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ally to set the quality, by subscribing to different layers 
of error correction. Further, the transmitter may send the 
same information on different streams at different times, 
so that a receiver can replace lost packets by subscribing 
to later streams; and this may form a hybrid scheme with 
forward error correction. 
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RECEIVER-DRIVEN LAYERED ERROR CORRECTION MULTICAST OVER THE INTERNET 

5 BACKGROUND OF THE INVENTION 

Technical Field: 

The invention is related to a system and process for correcting errors and 
10 losses occurring during a receiver-driven layered multicast of real-time media over a 
heterogeneous packet network such as the Internet. 

Background Art: 

15 Real-time media, such as radio and television programs, are broadcast from a 

single sender to multiple, geographically distributed receivers, who have all "tuned" 
to that sender. Commonly, the signals are broadcast from the sender by a terrestrial 
antenna, but satellite and wired solutions also exist. For example, in cable TV, the 
signals are broadcast from a sender by propagating a voltage along a coaxial cable 

20 to receivers connected to the cable. 

It is also possible to use the Internet infrastructure to broadcast audio and 
video information. This is typically accomplished using the Internet Protocol (IP) 
Multicast mechanism and its associated protocols. An Internet broadcast (or more 
25 properly, "multicast") is provided to the set of receivers who have first "subscribed" to 
the information. Specifically, through an announcement mechanism, such as a web 
page, a broadcaster announces the IP multicast group address to which it will send a 
particular broadcast. The multicast group address is just a special case of an 
ordinary IP address. However, unlike an ordinary address which is used to identify 
the "location" of a receiver where data is to be sent, a multicast group address is 
used by routers in the network to identify data being transmitted on the network as 
part of the broadcast, so that it can be routed to a subscribing receiver (who will have 
a completely different address). The receiver's address is not included in the 
broadcasted information. 
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A receiver subscribes to the broadcast by notifying the network that it wishes 
to "join" the multicast group. The subscriptions cause various routers in the network 
to update their states, to ensure that the multicast information eventually reaches the 
subscribers. At some point the sender begins to send packets to the specified 
5 address. When a router receives a packet with that address, it sends copies of the 
packet through each outgoing interface that leads to a subscriber. This causes the 
packets to reach the subscribers at some point, albeit with the inevitable packet loss 
due to network congestion and buffer overflow. 

10 At a later point in time a receiver may unsubscribe for reasons that will be 

discussed later. This also causes the routers to update their states. If a router no 
longer has subscribers downstream from an interface, it stops copying the multicast 
packets to that interface. If a router no longer has subscribers downstream from any 
of its interfaces, then the router itself unsubscribes from the multicast group, and 

15 hence no longer receives (from upstream routers) multicast packets addressed to 
that group. This process is reversible, and dynamic. Receivers may subscribe and 
unsubscribe as many times as desired. Thus, information is propagated through the 
network only as necessary to reach currently subscribing receivers. The processes 
of subscribing and unsubscribing takes only fractions of a second, thereby network 

20 bandwidth is not wasted unnecessarily. 

In the Internet, the channels between the sender and each receiver vary 
dramatically in capacity, often by two or three orders of magnitude. These 
differences in capacity exist because the data transmission rates associated with the 

25 connections to a particular receiver can vary (e.g., phone line capacity, LAN and/or 
modem speeds). This heterogeneity in capacity can cause problems in the context 
of an Internet broadcast of real-time audio and video information. For example, a 
particular receiver may not have the bandwidth available to receive the highest 
quality transmission that a broadcaster is capable of providing. One early attempt to 

30 cope with this problem involved broadcasting the audio and video data at different 
transmission rates to different multicast group addresses, with the quality being 
progressively better in the data broadcast at the higher rates. The receiver then 
subscribed to the transmission that suited its capability. However, this solution was 

2 
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very bandwidth intensive as the same information (and more) had to be repeated in 
each channel. To overcome this problem, an Internet broadcast can be transmitted 
via a "layered multicast". In a layered multicast, audio and video information is 
encoded in layers of importance. Each of these layers is transmitted in a separate 
5 data stream. A data stream is a sequence of packets all transmitted to the same 
multicast group address. The base layer is an information stream that contains the 
minimal amount of information, for the least acceptable quality. Subsequent layers 
enhance the previous layers, but do not repeat the data contained in a lower layer. 
Thus in order to obtain the higher quality, a receiver must subscribe to the lower 

10 layers in addition to the higher layers that provide the desired quality. For example, 
a video signal can be layered into packetized data streams of 8 Kbps (thousand bits 
per second). Each stream is sent to a different multicast group address. A receiver 
can subscribe to as many streams as it wants, provided the total bandwidth of the 
streams is not greater than the bandwidth of the most constrained link in the network 

15 between the sender and the receiver. For example, if the receiver is connected to 
the Internet by a 28.8 Kbps modem, then it can feasibly subscribe to one, two, or 
three 8 Kbps video layers. If it subscribes to more than three layers, then congestion 
will certainly result and many packets will be dropped randomly, resulting in poor 
video quality. 

20 

j 

It has been proposed that the congestion problem be addressed using a 
"Receiver-driven Layered Multicast" (RLM) scheme, where each receiver attempts to 
optimize its received quality by subscribing to as many layers as possible without 
incurring substantial congestion and loss. It does this by "test joins," in which the 

25 receiver tentatively joins, or subscribes to, the multicast group containing the next 
layer. If performance improves, the test join is made permanent. Otherwise, the 
layer is dropped, i.e., the receiver unsubscribes from the multicast group. In 
addition, if performance degrades at any point during the multicast (due to 
congestion in the network), the topmost layer is dropped. However, in complex 

30 network environments such as the Internet, there is often congestion along the path 
between the sender and receiver that is "ambient" in the sense that it is due to cross 
traffic between other senders and receivers. Therefore, it is not always possible to 
eliminate all congestion along the path from a sender to a receiver by cutting back on 
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the rate of transmission between the sender and receiver i* k m 

the multicast Thic n , receiver, i.e., by dropp.ng layers of 

anemo t h * Pr ° b ' em beCaUSe in RLM receiver 

attempts to drop recast groups until there is ,itt, e or no loss. When there is 

amb, nt congestion, this resuits in the receiver subscribing to few tfa y a re 
s wh,ch ,n turn results in sub-optima, video and audio quality ' 

* CCOrdin ^ there is a need for a system and process that can overcome the 
congestion ,ssue and its concomitant packet losses, without eliminating so m anv 



DISCLOSURE OF THE INVENTION 

' 5 RLM JT PreSent inVen "° n a ° COrnPliSheS ,HiS teSk by ' in — -~ 9 
"" h " m ° re <* — -rrectlon Nation. This allow! each 

n.ormafon by subscribing to a, leas, one error correction layer. Thus a unk,ue 
^ rece.ver-dnven, iayered. error correction muKicas, system and process Is 

Ideally, each source layer in a RLM would have one or more multicast error 
~ ata streams (I.e.. layers, associated therewith. Each o, the e" 
correcon layers would contain informalion that can be used ,o repiace lost packets 
from the associated source layer. More than one error correction ayer is prop I 

J; r; r ror correc,ion m in - — ~ r 

eplace the packets los, in the assodated source stream may themselves be lost in 
a s~: r TheSe ,0St eTOr ^ - - ^ - * « .o 

: 0 r:~rre;aie n d d : z: a " (or an acc ~ ~> - - 

» layers availabte. "" more addi «<™' error correction 

correct de °' Si ° n " " ^ a " d **°*ated error 

correct, on layens to subscribe to a, one time is in essence based on the inhere* 
Packet loss rate * the network connection and the maximum bandwidth av e to 
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the receiver. The idea is to subscribe to as many of the source layers as possible or 
desired, while leaving enough bandwidth available to also subscribe to the number of 
error correction layers for each source layer that will compensate for the inherent 
packet loss rate of the connection and provide an acceptable audio and video 
5 quality. It should be noted that there is also an option not to subscribe to any error 
correction layers. For example, in networks where the inherent packet loss rate is 
very low, the optimum quality might be obtained by subscribing to additional source 
layers and no error correction layers. 

10 The decision logic could be viewed as a one time decision made prior to 

receiving the broadcast. However, the packet loss rate, or even the available 
bandwidth could fluctuate during the broadcast. Accordingly, the decision logic could 
also be implemented dynamically in that the number of source layers and associated 
error correction layers subscribed to is reevaluated on a periodic basis to ensure the 

15 optimum quality is maintained throughout the broadcast. It should be noted that this 
may entail unsubscribing to the topmost layer (or layers) at least temporarily to 
maintain a desired audio and video quality. 

Any audio and/or video layering process currently used in RLM could feasibly 
20 be employed in conjunction with the above-described system and process to create 
the source layers. Likewise, any conventional process currently used in RLM can be 
adopted for encoding/decoding and packetizing/unpacketizing the audio and/or video 
signal in the present invention. Further, in regard to the error correction layers, it is 
possible to use any currently existing error correction technique appropriate for 
25 packetized audio and video signals. For that matter, any combination of such 
existing error correction techniques could be employed. In addition, two specific 
error correction methods or techniques, or a combination of these two methods, be 
employed as discussed below. 

30 The first of these specific error correction methods is a unique adaptation of 

an existing process known as Forward Error Correction (FEC). In essence, the FEC 
technique involves encoding the transmission data using a linear transform which 
adds redundant elements. The redundancy permits losses to be corrected because 
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any o, the origina, data events can be derived from the encoded elements Thus 
as long as enough of the encoded data elements are received so as to equal the ' 

SoecT ^ Z 0r r, a ' d3,a e ' emen,S - * 15 P ° SSible 10 dS * e a " - «-**- cements 
Specficaily. the FEC technique is adapted to the present invention by producing 

party layers tha, go aiong with each source tayer. To term the party ,ayers the 
packets ir , a source tayer are partitioned into k packets per block, terming ter each 
block a k by m matrix of bytes. X. Then a systematic, rate-compatible forward error 
correctton code is applied to each block, to produce an n by m matrix o, bytes Y 
write terms n-k parity packets (in addition te the origina, k source packets). Bad, o, 
hese n-k parity packets is assigned te a different parity stream or layer. Each parity 
ayer . thentransmtted in a separate stream te a deferent mutticas. group address 
in «h,s way, both the source and parity information is .ayered. For eacri source layer 
teere are n-k parity layers. By having many source layers, and many parity layers fo'r 
each source layer, each receiver is ab,e to independently subscribe te the optima, 
number of source layers, and the optima, number of parity layers for each source 
layer, so as to maximize quality for a given transmission rate. 

The second of the aforementioned error correction techniques is new but 
loosely based on an existing process called automatic repeat request (ARQ)' 
protocol. ,n the ARQ protoco,, the receiver is ab,e to identify (e.g., via missing 
packet sequence numbers, which packets have been lost in transmission and 
request retransmission of the test packets from the sender. This process would 
typ,ca.ly be impractical however in the context of a IP multicast on a large netwo* 
such as the Internet because the broadcaster would be overwhelmed by 
retransmission requests from the receivers. ,n the aforementioned new error 
correct™ process, dubbed pseudo-ARQ, the receivers do not request 
retransmission of specific lost packets, instead, the broadcaster sends no, only the 
source packets in a primary stream, bu, also sends delayed versions thereof in one 
of more redundant streams to different multicast group addresses. In this way the 
tece.ver can subscribe as necessary to one or more of the delayed streams to pick 
up those packets needed to replace lost packets in the primary streams 
Speoflcally, in pseudo-ARQ. the sender multicast* the source packets in a primary 
stream, and also multioasts delayed versions of the source packets in one or more 
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redundant streams. If a receiver loses a packet from the primary stream, then it has 
the opportunity to subscribe to the first of the redundant streams, in an attempt to 
receive the packet again in a second transmission. If that fails too, then the receiver 
has the opportunity to subscribe to the second redundant stream, and so forth, until 
5 either the receiver recovers the desired packet, or there are no more streams to 
subscribe to. 

The two aforementioned error correction processes can also be combined to 
form an advantageous third process. This hybrid FEC/pseudo-ARQ process differs 

10 from the pure FEC in that all or some of the parity streams are progressively 

delayed. This permits the receiver to subscribe to additional parity information on 
demand. In one version of the hybrid error correction process, one or more 
undelayed parity layers are subscribed to and the first part of the process is identical 
to that described earlier in connection with the pure FEC procedure. However, if the 

15 number of parity packets lost in transmission results in fewer parity packets than 
missing source packets in a block of source layer packets, these parity packets can 
be obtained by subscribing to a delayed parity stream. 

In another version of the hybrid process, all the parity streams are delayed 
20 and a receiver subscribes as needed to the streams to obtain parity packets that can 
be used to compute replacement source layer packets. This version is similar to the 
pseudo-ARQ method except parity packets are transmitted instead of copies of the 
source layer packets. This has advantages because rather than sending a copy of 
every source stream packet making up a block in an error correction stream, only 
25 one parity packet need be sent which multiple receivers can use to recover different 
source packets belonging to the block associated with the parity packet. This makes 
for an efficient use of shared network bandwidth. For example, two receivers may 
have a different loss pattern (of two losses) in a particular block of source data. But 
both receivers can still subscribe to the same parity packet associated with that block 
30 to try to recover the losses. Because the parity streams contain parity packets rather 
than a copy of a source packet, these packets can be used to recover different 
source packets in the same block. This reduces the network bandwidth shared by all 
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reivers, because the receivers can capture the same packet, rather than many 
Afferent packets, to recover different loss patterns. 

A further refinement of the hybrid error correction involves collapsing groups 
of pamy streams into a single combined pauty stream. The advantage of collapsing 
mutuple streams into a single stream is that the receiver can then subscribe and 
unsubscribe to the single stream as necessary to obtain the packets it needs, rather 
than subscribing and unsubscribing to multiple streams. Thus, a receiver can capture 
mutuple parity packets associated with the same block in me same collapsed parity 
stream to compute replacements for more than one tost source packet in mat block 
of source data. This decreases the memo-y requirements in the network associated 
with keeping so many connections open. 

In addition to the just described benefits, other advantages of the present 
-nvenfon will become apparent from the detailed description which follows 
hereafter when taken in conjunction wrth the drawing figures which accompany it 



BRIEF DESCRIPTION OF THE DRAWINGS 

The specific features, aspects, and advantages of the present invention will 
become better understood with regard to the fo.lowing description, appended Cairns 
and accompanying drawings where: 

FIG. 1 is a diagram depicting a general purpose computing device constituting 
an exemplary system for implementing the present invention. 

FIG. 2 is a diagram depicting an exemplary architecture for the receiver- 
based, layered, error correction multicast system according to the present invention. 

FIGS. 3A and 3B are block diagrams of an overall process for practicing the 
rece,ver-based. layered, error correction multicast system according to the present 
invention. 



8 



WO 00/761 1 3 PCT/USOO/1 4060 

FIG. 4 is a block diagram of a process for monitoring available bandwidth and 
packet loss rates, and reacting to significant changes thereto, which is performed in 
conjunction with the overall process of Figs. 3A and 3B. 

5 FIG. 5 is a block diagram of a process for practicing the receiver-based, 

layered, error correction multicast system according to the present invention wherein 
parity layers to be used in the error correction are generated using an adapted 
Forward Error Correction (FEC) technique. 

10 6 is a diagram depicting the relative flow of data packets for a source 

stream and nine associated parity streams generated via the FEC technique of Fig. 

5. 

FIG. 7 is a block diagram of a process for practicing the receiver-based, 
15 layered, error correction multicast system according to the present invention wherein 
a receiver subscribes to the parity streams generated by the process of Fig. 5 and 
uses the parity streams to replace source layer packets lost during transmission. 

FIGS. 8A and 8B are block diagrams of a process for practicing the receiver- 
20 based, layered, error correction multicast system according to the present invention 
wherein error correction layers are generated using a pseudo-Automatic Repeat 
Request (ARQ) technique, and wherein a receiver subscribes to the error correction 
streams and uses them to replace source layer packets lost during transmission. 

25 FIG. 9 is a diagram depicting the relative flow of data packets for a primary 

source stream and two delayed error correction streams generated via the pseudo- 
ARQ technique of Figs. 8A and 8B. 

FIG. 10 is a diagram depicting the relative flow of data packets for a source 
30 stream, three parity streams, and six delayed parity streams generated via a hybrid 
FEC/pseudo-ARQ error correction technique. 
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FIG. 1 1 is a diagram depicting the relative flow of data packets for a source 
stream, three parity streams, and two collapsed delayed parity streams generated 
via a hybrid FEC/pseudo-ARQ error correction technique. 

5 FIGS. 12A and 12B are block diagrams of a process for practicing the 

receiver-based, layered, error correction multicast system according to the present 
invention wherein parity layers to be used in the error correction are generated using 
a hybrid FEC/pseudo-ARQ error correction technique. 

10 FIGS. 13A and 13B are block diagrams of a process for practicing the 

receiver-based, layered, error correction multicast system according to the present 
invention wherein a receiver subscribes to the parity streams generated by the 
process of Figs 12A and 12B and uses the parity streams to replace source layer 
packets lost during transmission. 

15 

MODES FOR CARRYING OUT THE INVENTION 

In the following description of the preferred embodiments of the present 
20 invention, reference is made to the accompanying drawings which form a part 

hereof, and in which is shown by way of illustration specific embodiments in which 
the invention may be practiced. It is understood that other embodiments may be 
utilized and structural changes may be made without departing from the scope of the 
present invention. 

25 

Fig. 1 and the following discussion are intended to provide a brief, general 
description of a suitable computing environment in which the invention may be 
implemented. Although not required, the invention will be described in the general 
context of computer-executable instructions, such as program modules, being 
30 executed by a personal computer. Generally, program modules include routines, 
programs, objects, components, data structures, etc. that perform particular tasks or 
implement particular abstract data types. Moreover, those skilled in the art will 
appreciate that the invention may be practiced with other computer system 
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configurations, including hand-held devices, multiprocessor systems, 
microprocessor-based or programmable consumer electronics, network PCs, 
minicomputers, mainframe computers, and the like. The invention may also be 
practiced in distributed computing environments where tasks are performed by 
5 remote processing devices that are linked through a communications network. In a 
distributed computing environment, program modules may be located in both local 
and remote memory storage devices. 

r 

With reference to Fig. 1, an exemplary system for implementing the invention 

10 includes a general purpose computing device in the form of a conventional personal 
computer 20, including a processing unit 21, a system memory 22, and a system bus 
23 that couples various system components including the system memory to the 
processing unit 21 . The system bus 23 may be any of several types of bus 
structures including a memory bus or memory controller, a peripheral bus, and a 

15 local bus using any of a variety of bus architectures. The system memory includes 
read only memory (ROM) 24 and random access memory (RAM) 25. A basic 
input/output system 26 (BIOS), containing the basic routine that helps to transfer 
information between elements within the personal computer 20, such as during 
start-up, is stored in ROM 24. The personal computer 20 further includes a hard 

20 disk drive 27 for reading from and writing to a hard disk, not shown, a magnetic disk 
drive 28 for reading from or writing to a removable magnetic disk 29, and an optical 
disk drive 30 for reading from or writing to a removable optical disk 31 such as a CD 
ROM or other optical media. The hard disk drive 27, magnetic disk drive 28, and 
optical disk drive 30 are connected to the system bus 23 by a hard disk drive 

25 interface 32, a magnetic disk drive interface 33. and an optical drive interface 34, 
respectively. The drives and their associated computer-readable media provide 
nonvolatile storage of computer readable instructions, data structures, program 
modules and other data for the personal computer 20. Although the exemplary 
environment described herein employs a hard disk, a removable magnetic disk 29 

30 and a removable optical disk 31, it should be appreciated by those skilled in the art 
that other types of computer readable media which can store data that is accessible 
by a computer, such as magnetic cassettes, flash memory cards, digital video disks, 
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(ROMs), und the ,i k e. may also be J£Z ~ ° " mem0ri6S 

usea in the exemplary operating environment, 
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network (WAN) 52. Such networking environments are commonplace in offices, 
enterprise-wide computer networks, intranets and the Internet. 

Wt-en used in a LAN networking environment, the personal computer 20 is 
5 connected to the local network 51 through a network interface or adapter 53. When 
used in a WAN networking environment, the personal computer 20 typically includes 
a modem 54 or other means for establishing communications over the wide area 
i network 52, such as the Internet. The modem 54, which may be internal or external, 

is connected to the system bus 23 via the serial port interface 46. In a networked 
10 environment, program modules depicted relative to the personal computer 20, or 
portions thereof, may be stored in the remote memory storage device. It will be 
appreciated that the network connections shown are exemplary and other means of 
establishing a communications link between the computers may be used. 

1 5 The exemplary operating environment having now been discussed, the 

remaining part of this description section will be devoted to a description of the 
program r nodules embodying the invention. 

1.0 Receiver-Driven Layered Error Correction. 

20 

The present invention accomplishes the aforementioned task of overcoming 
the congestion issue and its concomitant packet losses by, in essence, augmenting 
RLM with one or more layers of error correction information. This allows each 
receiver to separately optimize the quality of the received audio and video 
25 information by subscribing to at least one error correction layer. Thus, a unique 
receiver-driven, layered, error correction multicast system and process is created. 

It is noted that the term "layered" in the context of RLM refers to the fact that a 
base data stream is provided along with additional hierarchical enhancement layers 
30 which respectively add more and more data to progressively improve the quality of 
the signal eventually reconstructed from the layers. The aforementioned error 
correction data streams are not technically layers because each stream associated 
with a particular source layer provides the same error correction information. 

13 
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However, as viewed between source layers, the error correction data streams do 
prov,de different error correction information. Thus, while not technically correct the 
error correction data streams will alternately be referred to as data or information 
streams, as well as layers in the description of the present invention. 

Each source layer in a RLM would have at least one broadcasted error 
correction layers associated therewith, and ideally multiple error correction layers 
Each of the error correction layers contains information that can be used, alone or in 
conjunction with other error correction layers, to replace lost packets from the 
associated source layer. The use of more than one error correction layer is 
preferred as some of the error correction packets contained in the error correction 
layer, which are needed to replace the packets lost in the associated source stream 
may themselves be lost in transmission. These lost error correction packets can be 
P.cked up by subscribing to a second error correction layer, and so on until all (or an 
acceptable number) of the source packets have been replaced, or there are no more 
additional error correction layers available. In addition, in some circumstances as 
will be explained in more detail later, there may be too many missing source layer 
packets in a portion of the source data stream for a single error correction layer to 
prov.de all the replacements needed. Thus, having multiple error correction layers 
available vor a given source layer improves the chances that all the missing source 
data packets in that layer can be replaced. 

The present invention can be implemented using the system architecture 
shown in Fig. 2. The depicted architecture is divided into two main parts a 
broadcaster 200 and receiver 202, which are connected across a heterogeneous 
packet network 204, such as the Internet. The broadcaster 200 is associated with 
the source of the IP multicast and the receiver 202 is associated with a network user 
rece.v,ng the multicast. In the broadcaster 200, an audio and/or video signal 206 
containing information that is to be broadcast is input into a encoder and packetizer 
module 208. This module 208 is responsible for creating the aforementioned source 
layers by encoding and packetizing the signal data into a series of data streams. 
This results in an output having a base layer 210, as well as one or more 
enhancement layers 212. The base layer 210 and each enhancement layer 212 are 

14 
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then processed by an error correction module 214 to create one or more error 
correction layers from the associated source layer. Each source layer and its 
associated error correction layers are then transmitted onto the network to separate 
multicast group addresses, as described previously. By way of an example, Fig. 2 
5 depicts a source layer 216 and two error correction layers 218 being sent onto the 
network from each error correction module 214 . Of course, more or fewer error 
correction layer streams could be sent as desired. Referring now to the receiver 
202, some of the transmitted source layer streams, as well as their associated error 
correction layer streams, are received by a subscription module 220. This 

10 subscripti :>n module 220 subscribes to or unsubscribes from the various multicast 
group addresses so that the desired number of source and error correction layers 
are received. The subscription and un-subscription process proceeds in the network 
as described earlier in connection with the explanation of the RLM scheme. In the 
depicted example, the subscription module 220 is shown as subscribing to the base 

15 source layer 216 and two of its associated error correction layers 21 8, as well as the > 
first enhancement source layer 222. Each received source layer and its associated 
error correction layers are fed into a recovery module 224 from the subscription 
module 220. The recovery module 224 uses the error correction layers (if present) 
to replace as many of the packets missing from the source layer as possible. The 

20 recovered source layers are then input into an unpacketizer and decoder module 
230. In the depicted case, the recovered base source layer 226 and the first 
enhancement layer 228 are fed into this module 230. The unpacketizer and decoder 
module 230, as its name implies, unpacketizes each of the recovered source layers 
and decodes the layers to reconstruct the audio and/or video signal. The signal 232 

25 is then output for further processing and use. 

The decision as to how many broadcast source layers and associated error 
correction layers to subscribe to at one time is in essence based on the inherent 
packet loss rate of the network connection and the maximum bandwidth available to 
30 the receiver. The idea is to subscribe to as many of the source layers as possible or 
desired, while leaving enough bandwidth available to also subscribe to the number of 
error correction layers for each source layer that will compensate for the inherent 
packet loss rate of the connection and provide an acceptable audio and video 
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quality. It should be noted that there is also an option not to subscribe to any error 
correction layers. For example, in networks where the inherent packet loss rate is 
very low. the optimum quality might be obtained by subscribing to additional source 
layers and no error correction layers. 

The decision logic could be viewed as a one time decision made prior to 
receiving the broadcast. However, the packet loss rate, or even the available 
bandwidth could fluctuate during the broadcast. Accordingly, the decision logic could 
also be implemented dynamically in that the number of source layers and associated 
error correction layers subscribed to would be reevaluated on a periodic basis to 
ensure the optimum quality is maintained throughout the broadcast. It should be 
noted that this may entail unsubscribing to the topmost layer (or layers) as necessary 
to maintain a desired audio and video quality. The decision as to how many source 
and error correction layers to subscribe to at any one time is made in the 
subscription optimization module 234. This module 234 instructs the subscription 
module 220 how many source and error correction layers to subscribe to at any 
given time. As described above, this decision is based on the estimated maximum 
transmission rate as well as the packet loss rate. The necessary rate information is 
derived from inputs received from the recovery modules 224, and the unpacketizer 
and decoder module 230, as shown in Fig. 2. 

Referring to Fig. 3A and 3B, a preferred process for implementing the 
receiver-driven, layered, error correction multicast system will now be described. 
The first two steps occur at the broadcast source. In step 300, one or more error 
correction streams are created from each source layer. The procedures that can be 
used to accomplish this task will be described in more detail later. Once the error 
correction streams have been created, they are broadcast along with the source 
layers (step 302). Each of the streams has a separate and unique multicast group 
address associated with it, as do the source layers. 

The process now turns to the receivers of the broadcast. Each receiver first 
determines, prior to the broadcast, the currently available bandwidth and the packet 
loss rate being exhibited by received data (step 304). This information, along with 
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the quality desired for the received broadcast, is employed in step 306 to select the 
source layers and associated error correction streams that are to be received to 
either optimize the quality or obtain the desired quality level. Finally, in step 308, the 
selected source layers and error correction streams are subscribed to by contacting 
5 the network and specifying which multicast group address it wants to receive. 

The process then shifts to the network. In step 310. the network reconfigures 
to route the subscribed to data to the subscribing receiver's network address. This is 
accomplished by each router between the broadcast source and the receiver 

10 updating its routing table to include an output to either the receiver or the next closer 
router to the receiver. The network, via its routers, then recognizes the multicast 
group address of the subscribed to source layers and error correction streams in the 
data being sent over the network (step 312). These addresses are embedded in 
each data packet making up the source layers and error correction streams. The 

15 recognized source layers and error correction streams are then routed to the 
receiver in step 314. 

The process then shifts back the receivers. Each receiver uses the received 
error correction streams to replace data packets of the associated source layer that 
20 were lost during transmission (step 316). This produces a recovered source layer. 
Finally, in step 318, each receiver reconstructs the video and/or audio signal from 
the recovered source layers. 

The foregoing procedure assumes that the available bandwidth and packet 
25 loss rate will stay constant throughout the broadcast. However, this may not be the 
case, and the quality of the received video and audio could be degraded if the 
bandwidth drops or the loss rate increases significantly. Further, the bandwidth 
might increase, or the loss rate decrease. This would present an opportunity for a 
receiver to subscribe to more source layers or error correction layers (or both) to 
30 improve the quality. However, the aforementioned assumption would prevent a 
receiver from taking advantage of the opportunity to increase the quality of the re- 
constructed signal. Referring to Fig. 4, an optional procedure for monitoring the 
available bandwidth and packet loss rate during the broadcast and reacting to 
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techniques could be employed. However, it is preferred that two specific error 
correction methods, or that a combination of these two methods, be employed. 



The first of these methods is an adaptation of an existing process known as 
5 Forward Error Correction (FEC). This is an error control coding method used to 
obtain reliable communications in a system susceptible to losses, such as the 
Internet. In essence, the FEC technique involves encoding the transmission data 
using a linear transform which adds redundant elements. The redundancy permits 
tosses to be corrected because any of the original data elements can be derived 
10 from any of the redundant elements. Thus, as long as enough of the encoded data 
elements are received so as to equal the number of the original data elements, it is 
possible to derive all the original elements. 

The second of the aforementioned preferred error correction techniques is 

15 new, but loosely based on an existing process called automatic repeat request 
(ARQ) protocol. In the ARQ protocols, the receiver is able to identify (e.g., via 
missing packet sequence numbers) which packets have been lost in transmission, 
and request retransmission of the lost packets from the sender. This process is 
typically impractical however in the context of a IP multicast on a large network such 

20 as the Internet because the broadcaster would be overwhelmed by retransmission 
requests from the receivers. In the aforementioned new error correction process, 
dubbed pseudo-ARQ, the receivers do not request retransmission of specific lost 
packets. Instead, the broadcaster sends not only the source packets in a primary 
stream, but also sends delayed versions thereof in one or more redundant streams 

25 to different multicast group addresses. In this way, the receiver can subscribe as 

necessary to one or more of the delayed streams to pick up those packets needed to 
replace lost packets in the primary streams. Thus, whereas in standard ARQ, the 
receiver sends repeat requests directly to the sender, in pseudo-ARQ, the receiver 
sends subscribe and unsubscribe messages (i.e., "join" and "leave" messages) to 

30 the network. Hence, there is no repeat-request implosion problem at the sender. 
The system scales up to millions of receivers in exactly the same way that IP 
Multicast and its companion protocols scale up. In fact, only the ordinary multicast 
group "join" and "leave" messages are used to implement pseudo-ARQ. 
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These two preferred error correction processes can also be combined to form 
an effective third process Essentially, the combined process employs the 
aforementioned FEC coding techniques, except that the transmission of some or a„ 
of the redundant elements is deiayed. ,n this way a receiver can subscribe fo one of 
the delayed error correction streams to capture parity packets needed to compute 
replacements for lost source layer packets 

The two preferred error correction techniques, as well as the aforementioned 
combmed technique will now be described in detail in the following sections. 

2.0 Receiver-Driven Layered Error Correction Employing A Forward Error 
Correction (FEC) Technique. 
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Here. „ > K and y „ ... y „ is an encodjng Q# ^ ^ ^ ^ ^ ^ ^ ^ 

the ongina, source data, x, x, the encoded data y,... y „ (or -codeword-, is sent 

Since n > k. there is redundancy. This redundancy permits losses to be recovered 
and enors to be corrected. Only particular values of the codeword y, y„ are 
allowed. Because of this constraint, if any of the components of yare missing, they 
oan be replaced, provided a. leas, k of mem are actually received. The source data 
*' Xk Ca " ' hen ,ound ^ so ^ng the above linear equation. Note mat me 
elements y„ x, and a s are all elements of a finite field, such as bite or bytes, and tha, 
multiplication and division are carried out in that finite field. For example, in the 
binary field, addition is carried ou, modulo 2. The "rate" of the code is given by the 
ratio Un. The rate is the number of source symbols transmitted per encoded symbol. 
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The highe r the rate, the more source information can get through with a given 
transmission rate, but the lower the redundancy, hence the fewer error or losses that 
can be tolerated. In the context of the present invention, it is preferable to choose a 
code rate that will ensure the desired quality in reception in view of the anticipated 
packet loss rate of the network, assuming the transmission rate is not so large at that 
code rate that the available bandwidth of a typical receiver is unacceptably 
monopolized by the error correction stream. 
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Different FEC codes have different properties. FEC codes in which the 

source data appears directly in the codeword (e.g., y,=x, for /=1 k) are called 

"systematic" codes. FEC codes at different rates k/n 1t k/n 2 , ... that are nested (in the 
sense that for each source vector x u . . „x k , their n r dimensional codewords y 1t . . .y n1 , 
yi,...y„ 2 .... are prefixes of one another) are called "rate-compatible" codes. Thus a 
code such as the one shown in the equation above can give rise to a whole family of 
rate-compatible codes, simply by deleting some of the rows at the bottom of the 
matrix. 
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can represent the m bytes in each of n FEC packets. If the FEC code is systematic 
(i.e., the original source code is included therein), then the first k FEC packets will be 
identical to the k source packets. If the FEC code is rate-compatible, then FEC 
codes with increasing redundancy can be generated simply by adding "parity- 
packets to the first k, up to a total of n packets. 

As discussed previously, in standard RLM, the source is encoded in layers, 
and each layer is transmitted in a separate data stream. The present FEC error 
correction process augments RLM by producing error correction (or parity) layers 
that go along with each source layer. The general process of forming these parity 
layers for any one of the source layers is shown in Fig. 5. In step 500, the packets in 
a source layer are partitioned into k packets per block, forming for each block a k by 
m matrix of bytes, X (where m is the number of bytes in each packet). Then, in step 
502, a previously unselected block is selected. A forward error correction code such 
as that described above is applied to the chosen block in step 504, to produce an n 
by m matrix of bytes Y, which forms n-k parity packets (in addition to the original k 
source packets if the code is systematic) for that data block. Each of these n-k parity 
packets is then assigned to a different parity streamNn step 506. Thus, there will be 
n-k parity streams for every source layer. As indicated in step 508, if the currently 
selected block is not the last block in the multicast, then steps 502 through 506 are 
repeated for the next data block. Otherwise the process proceeds on to step 51 0 
where each parity stream is transmitted separately to a different multicast group 
address (as is the associated source layer). It is noted that the foregoing process 
can be accomplished .ahead of time and the parity streams produced stored until the 
time of the multicast. However, alternatively, the parity streams can be produced 
and multicast as each block of the source layer is multicast, thus resulting in a real- 
time generation of the parity streams and no need to store them. 

As a concrete example of the foregoing process, suppose the video source is 
encoded into 8 Kbps layers, such that each layer produces one packet per second, 
each packet containing 1 Kbytes of information. Suppose then that every three 
seconds, k=Z packets are blocked together into a 3 x 1 Kbyte matrix X. Then 
suppose that for each such block of packets, n =12 packets are produced (as a 12 x 
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1 Kbyte matrix Y). The first k=3 of these are source packets, while the last n-k=9 are 
parity packets (i.e., the FEC code used was of the systematic type). Each parity 
packet is transmitted in a separate stream, forming 9 different parity streams for that 
source layer, each having a bit rate of 2.67 Kbps (one 1Kbyte packet every 3 

5 seconds). If there are 10 source layers, and each source layer has 9 parity layers, 
then there are a total of (1+9) x 10 = 100 streams. If a receiver is connected over a 
28.8 modem, and the ambient packet loss rate is 50%, then it may decide to 
subscribe to two 8 Kbps source layers, plus three 2.67 Kbps parity layers for the first 
source layer and no parity layers for the second source layer (for a total of 24 Kbps). 

10 It is well known that allowing unequal error protection for each source layer can 
optimize quality. 



The diagram of Fig. 6 illustrates the streams associated with the base source 
layer in the above example. Each stream constitutes a sequence of packets 

15 transmitted to the same multicast group address. The source stream is segmented 
into blocks, with k=3 packets per block. The source packets in block b are labeled 
Si,. j, ar >d Stu, and tn e corresponding n-k=9 parity packets are labeled P bi1 , 
P b .9. In subsequent blocks, the source packets are designated by the labels S b+1i1 , 
S/, + t, 2 , and S/,+,,3, and so on, with the associated parity packets being labeled Pb+i,u 

20 Pfc+7,9, and so on. It is noted that the parity packets are shown staggered in time, 
however, this need not be the case. Each parity packet associated with a source 
layer block can be transmitted at anytime during the transmission of that associated 
source block. It is also noted that the patterned packets in Fig. 6 are the ones to 
which the receiver in the above example subscribes. 
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Referring to Fig. 7, the receiver's side of the process will now be described. 
The first step 700 involves the receiver subscribing to a source layer. It is noted that 
the following description will use just one subscribed to source layer as an example 
of the process. It should be understood that the process is identical for each source 
layer subscribed to by a receiver. In step 702, the receiver also subscribes to one or 
more parity streams associated with the source layer. Once the multicast begins, 
the receiver begins receiving source layer packets partitioned in to data blocks as 
discussed previously (step 704). In addition, the receiver receives parity packets 
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associated with the incoming source layer data block in the subscribed to parity 
stream or streams (step 706). Once all me source layer and parity stream packets 
for an .ncoming data block have been received (less any packets lost during 
transm,ssion). it is determined whether the total number of these packets is at least 
equal to the number of source packets transmitted for the block, as indicated in step 
708. Th,s can be accomplished, tor example, by employing the sequence numbers 
these packets typically have embedded wrthin the data making up the packet Thus 
rt ,s readily apparent which packets are missing and the number of source layer ' 
packets originally multicast can be computed by adding the number of missing 
packets to the number of packet received. If i, is determined that the total number of 
source layer and parity stream packets received for the incoming block equal or 
exceed the number of source packets transmitted for that block, then all the missing 
source layer packets in the block are recovered using the received parity packets 
(step 710). This is accomplished by solving Equation (1) for the particular FEC code 
employed. The missing source layer packets associated with the incoming data 
block are then replaced with the aforementioned recovered packets in step 712 If 
however, it is determined that the total number of source layer and p-arity stream ' 
packets received for the incoming block do not equal or exceed the number of 
source packets transmitted for that block, then the incoming block is sent on for 
further process without recovering any of its missing packets (step 714) The all or 
noth.ng nature of this process derives from the previous* discussed fact that if any 
of the components of y are missing, they can be replaced, provided at least k of 
them are actually received. This means that if enough source layer and parity 
stream packets are received, despite losses during transmission, to equal or exceed 
the number of source layer packets originally transmitted, then all of the missing 
source layer packets can be recovered. However, if the total source layer and parity 
streams packets received is less that the original number of source layer packets 
sent, none of the missing source layer packets can be recovered. 

The foregoing recovery and replacement process (of steps 704 through 714) 
.s repeated for each block of source layer packets received, as indicated by step 
716. Preferably, this is accomplished as each block arrives so as to minimize the 
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delay in reconstructing the video and/or audio signal from the source layers and to 
minimize the amount of data that needs to be stored. 



It is apparent from the foregoing process that by having many source layers, 
5 and many parity layers for each source layer, each receiver is able to independently 
subscribe to the optimal number of source layers, and the optimal number of parity 
layers for each source layer, so as to maximize quality for a given transmission rate 
and the packet loss rate of the network connection. 



10 3.0 Receiver-Driven Layered Error Correction Employing A Pseudo 
Automatic Repeat Request (Pseudo-ARQ) Technique. 

Automatic repeat request (ARQ) protocols are commonly used in lossy packet 
networks. The ubiquitous TCP/IP is an example of such a protocol. In ARQ 

15 protocols, the receiver is able to identify which packets have been lost in 

transmission. The receiver is then able to request that the sender repeat the lost 
information. Although it may take many re-transmissions, the lost information is 
eventually recovered. Hence, the overall communication is "reliable" (i.e., without 
loss) ARQ is a very efficient mechanism that makes optimal use of the capacity of 

20 the forward transmission channel in that information is repeated only as necessary. 
Furthermore, it adapts naturally to the channel capacity, whatever it may be. This 
contrasts with FEC codes, whose redundancy is chosen to match a particular loss 
rate. For example, an FEC code with 25% redundancy (n/k - 1.25) is approximately 
optimal for channels that lose 1 in every 5 packets. If this FEC code is used, and the 

25 losses of the channel changes over time, then the code is no longer optimal. It is 

sending either too much parity information, or too little. The overall communication is 
not reliable, because the FEC method may occasionally fail, if too many packets are 
lost within a single block. On the other hand, because ARQ protocols adapt to the 
channel capacity, they have unbounded delay, while FEC has a guaranteed delay 

30 before decoding is possible. Of course, if a bounded delay is required (such as for 
real-time video), ARQ can be used until the delay bound is reached, after which point 
the method fails. In this case, the overall communication may be unreliable. 
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For the above reasons, ARQ is usually preferred to FEC whenever it is 
available. Unfortunately. ARQ is not usually feasible in broadcast scenarios. In the 
IP Multicast scenario, a feedback channel is usually available from each receiver to 
the sender. However, it is usually considered not to be feasible for millions of 
receivers to feed back packet re-transmission requests to a single sender. Not only 
would hundreds of thousands of simultaneous feedback requests "implode" upon the 
sender, but also it would not be feasible for the sender to send out hundreds of 
thousands of individual (i.e., unicast) re-transmissions to the receivers. 

The present invention introduces a pseudo-ARQ for real-time multicast 
transmission. In pseudo-ARQ, the sender multicasts the source packets in a primary 
stream, and also multicasts delayed versions of the source packets in one or more 
redundant streams. If a receiver loses a packet from the primary stream, then it has 
the opportunity to subscribe to the first of the redundant streams, in an attempt to 
recover the packet. If that fails too, then the receiver has the opportunity to subscribe 
to the second redundant stream, and so forth, until either the receiver recovers the 
desired packet, or there are no more streams to subscribe to. Once the lost packet 
is recovered from a subscribed to error correction data streams, or if the packet 
could not be recovered from that stream, the receiver can unsubscribe from the 
stream to conserve the available bandwidth. An error correction data stream would 
then be re-subscribed to the next time a source data packet is discovered missing. 
This frees up bandwidth and can provide an opportunity to subscribe to an additional 
source layer if sufficient bandwidth exists. Alternatively, one or more of the 
redundant error correction data streams can be permanently subscribed to during 
the broadcast to ensure quick access to replacement packets, albeit at the expense 
of bandwidth. It is also noted that there are only a finite number of redundant 
streams because there is a delay bound before which the real-time media must be 
decoded for presentation to the user. 

The technique involving continuous subscribing and unsubscribing to error 
correction data streams will now be described in detail with reference to Fig. 8: It is 
noted that the process is described in the context of one of the source layers and its 
associated error correction layers, but is applicable to all the other source and error 
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correction layers as well. In step 800, the broadcaster multicasts a source layer of 
video and/or audio in a data stream. In addition, the broadcaster multicasts one or 
more delayed versions of the source layer as error correction layers (step 802). The 
delay period for the first error correction layer is chosen such that a receiver has time 
5 to detect the loss of an incoming data packet in the source layer and subscribe to 
one of the delayed error correction layers before a replacement packet in the error 
correction layer arrives. Likewise the delay period for each successive error 
correction layer is chosen so that a receiver has time to detect the loss of a needed 
replacement data packet in an error correction layer and to subscribe to the next 
10 subsequent error correction layer to obtain that data packet. 

The next step in the process involves a receiver subscribing to one of the 
source layers (step 804). All the receivers subscribe to the base layer (i.e., the 
primary data stream). A receiver may also subscribe to one or more of the 

15 enhancement source layers, as desired. Each receiver attempts to reconstruct a 
subscribed to source layer from the received packets. The first step 806 in this task 
is to monitor the incoming source layer to determine if a source layer packet has 
been lost during transmission (via missing packet sequence numbers, for example). 
As indicated in step 808, the monitoring procedure continues until a missing source 

20 data packet is discovered. At that point, the receiver subscribes to an error 
correction stream (step 810). Specifically, the receiver subscribes to the error 
correction stream currently having the least amount of delay of the available 
streams. Then, in step 812, the receiver attempts to capture the delayed version of 
the lost source layer packet from the subscribed to error correction stream. As 

25 indicated in step 814, if it is discovered that the needed replacement packet is not 
contained in the subscribed to error correction stream (such as might happen if the 
packet was lost during transmission), then it is determined whether there are any 
more error correction streams available for subscription (step 816). If there are more 
error correction streams available, then the receiver unsubscribes from the current 

30 stream ar.d steps 810 through 818 are repeated. This entails, among other things, 
subscribing to a different error correction stream - specifically, the stream having the 
least amount of delay in comparison to any of the other available streams (excluding 
the ones already used). If, however, the subscribed to error correction stream did 
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contain the missing packet, or if there were no other error correction streams 
available for subscription, then the process proceeds to step 820. In this step, the 
receiver unsubscribes from the currently subscribed to error correction stream and 
replaces the missing source layer packet with the identical error correction layer 
5 packet, if it was recovered. Finally, as indicated in step 822, the process of 

monitoring the incoming source layer and replacing lost source layer packets (i.e., 
steps 806 through 818) is continuously repeated, until such time as the broadcast 
ends. ? 

10 The diagram shown in Fig. 9 depicts the content of three pseudo-ARQ data 

streams as they are transmitted over time (i.e., one source or primary stream and 
two delayed error correction streams). In this diagram, the patterned packets are the 
packets to which the receiver subscribes at any given time, and the data packets are 
labeled according to time (e.g. S f is the packet received at time =0, S t +i is a packet 

15 received at time = 1 , and S t -i is a packet in a delayed stream representing a source 
data packet preceding the S f packet by one time period). The X'sjndicate packets 
that are lost in transmission. The receiver always subscribes to the packets in the 
primary stream. In addition, the receiver subscribes^and unsubscribes to the first 
delayed stream twice — once in an attempt to recover packet S t +2 and once to 

20 recover packet S, +4 . The receiver also subscribes and unsubscribes to the second 
delayed stream once to recover packet S f+2 which was missing from the first error 
correction data stream. Meanwhile other packets are still being transmitted in the 
primary stream. Note that in some time intervals, the receiver subscribes to multiple 
streams simultaneously. For a fixed-rate communication channel (such as a 

25 modem), this might seem problematic. In practice, however, the channel is shared 
by many streams (e.g., for each source layer). A conventional overall rate-control 
mechanism, such as that employed in standard RLM, is employed to ensure a 
constant bit rate through the channel, possibly by adding or dropping less important 
layers. For example, a higher-level source layer may have to be dropped (i.e., 

30 unsubscribed from) in order to subscribe to an error correction layer associated with 
a lower-level source layer to replace lost data packets in that lower-level layer. Of 
course, the higher level source layer could be re-subscribed to once the lost packet 
is retrieved. 
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It is noted that in the foregoing example there were two error correction layers 
available. However, it is not intended to limit the invention to two layers. Rather, 
one layer could be employed, or more than two with the maximum number limited 
only by the maximum acceptable delay. If the missing source layer packet cannot be 
5 retrieved from the available error correction layers, then the packet is simply not 
replaced and the video and/or audio signal is reconstructed without it, albeit at lower 
quality. 

4.0 Receiver-Driven Layered Error Correction Employing A Hybrid FEC 
10 and Pseudo-ARQ Technique. 

Pseudo-ARQ and FEC techniques can be advantageously combined. This 
can be done as follows. As in FEC, the b'th block of k source packets, say S bi1 , .... 
S b .k, can be protected with n-k parity packets, say P br1 , .... P b ,„. k . However, these 
15 parity packets can be partitioned into "waves" with different delays, as in pseudo- 
ARQ. Each wave could include any desired number of parity streams, each of which 
would exhibit the same delay. For example, suppose k = 3 and n = 12, as in the 
FEC example above. Then the packets can be arranged in waves as depicted in the 
diagram of Fig. 10. 

20 

In the diagram, the heavily textured parity packets (P bj1 , ...,P bi3 ) are in the first 
wave of packets for a block (S b , 1f ...,S b , 3 ), the stippled parity packets (P bi4 , ...,P bi6 ) 
are in the second wave of packets for the same block, and the medium textured 
parity packets (P bJ , .. v P b , 9 ) are in the third wave of packets for this block. It is noted 
25 that the parity packets associated with a particular source layer block (e.g., P bi1 , 
...,P b ,6) are shown staggered in time in Fig. 10 between parity streams in the same 
wave. However, this need not be the case. Each parity packet associated with the 
same block in a wave could also be transmitted at the same time if desired. 

30 The hybrid FEC/pseudo-ARQ process in the depicted example differs from 

the pure FEC example simply in that streams 4-6 are delayed by three packets and 
streams 7-9 are delayed by six packets. This permits the receiver to subscribe to 
additional parity information in a later wave if the number of parity packets lost in the 
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current wave results in fewer parity packets than missing source packets. As 
described previously, it does not matter which parity packets associated with a block 
of source data are used to compute replacements for source layer packets missing 
in that block. However, the total number of source layer and parity stream packets 
received must still at least equal the number of source layer packets transmitted for 
each data block, or none of the missing packets can be recovered. Therefore, the 
receiver is able to pick up missing parity packets for a block on demand in delayed 
parity stream rather than subscribing to many parity streams throughout the multicast 
to ensure enough are received to recover all the missing source layer packets - a 
significant advantage over using FEC techniques alone. 

Another major advantage of the above hybrid FEC/pseudo-ARQ scheme, this 
time over a pure pseudo-ARQ scheme, is that rather than sending a copy of every 
source stream packet making up a block in an error correction stream, only one 
parity packet need be sent which multiple receivers can use to recover different 
source packets belonging to the block associated with the parity packet. This makes 
for an efficient use of shared network bandwidth. For example, two receivers may 
have a different loss pattern (of two losses) in a particular block of source data. But 
both receivers can still subscribe to the same parity packet associated with that block 
to try to recover the losses. Because the parity streams contain parity packets rather 
than a copy of a source packet, these packets can be used to recover different 
source packets in the same block. This reduces the network bandwidth shared by all 
receivers, because the receivers can capture the same packet, rather than many 
different packets, to recover different loss patterns. In an extreme case, where the 
block length is long (say 100 packets), all receivers having only a few losses in the 
long block can recover the losses by capturing the same few parity packets, 
regardless of where the losses occurred in the block. 

Referring now to the d.agram of Fig. 1 1 , a further refinement of the combined 
error correction process is shown. Specifically, the last six parity streams are 
collapsed into two parity streams. The advantage of collapsing multiple streams into 
a single stream is that the receiver can then subscribe and unsubscribe to the single 
stream as necessary to obtain the packets it needs, rather than subscribing and 
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unsubscribing to multiple streams. This efficiency comes at the price though in that 
the subscription and unsubscription operations must be accomplished quickly to 
ensure the parity packets associated with a particular source data block being 
processed are captured. However, if the receiver is capable of quickly capturing the 
5 required packets, subscribing to collapsed error correction streams can be quite 
advantageous. 

Referring to Fig. 11, an example of the use of the hybrid FEC/pseudo-ARQ 
error correction process will now be given. In the diagram, the textured packets are 

10 the packets to which the receiver subscribes at any given time. It is noted that the 
block nomenclature used in connection with the diagrams of Figs. 6 and 9 is 
employed here as well, and the X's still indicate packets that are lost in transmission. 
In this example, the receiver always subscribes to the packets in the source stream 
and in parity stream 1. As it requires one parity packet associated with a particular 

15 block of the source stream to compute each replacement for a lost packet from that 
stream, parity packet P^from parity stream 1 can be used to compute a 
replacement for source stream packet S bt1 . Likewise, parity packets P b+1t1 and P b + 2 ,i 
from parity steam 1 can be used to compute replacements for lost source stream 
packets S b + 1t1 and S b+2 ,i , respectively. However, there are no parity packets in 

20 parity stream 1 to use in computing replacements for the lost source stream packets 
S bt3 and S b + 1t2 . It will be remembered that the total number of source layer and parity 
packets associated with a source layer block that were received must equal or 
exceed the number of source layer packets transmitted, or none of the missing 
source layer packets can be recovered. Therefore, the receiver subscribes and 

25 unsubscribes to parity stream 4-6 twice. It subscribes a first time in an attempt to 
receive parity packet P bt4 to compute a replacement for lost source stream packet 
S bt3 . In addition, it subscribes a second time to capture the parity packet P b +i, 4 which 
can be used to compute a replacement for source stream packet S b + 2t1 . Next, since 
the attempt to capture parity packet P bf4 was unsuccessful as it was missing from 

30 the combined parity stream 4-6, the receiver also subscribes and unsubscribes to 
the second delayed parity stream 7-9 once to receive packet P bJ which can be used 
to compute a replacement for source stream packet S bt 3 . This process can be 
repeated using additional collapsed (or uncollapsed if that is the only type available) 
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packets associated with a particular source layer block will have to be staggered in 
time so that they can be transmitted in sequence in the collapsed stream. These 
collapsed streams can be generated by combining already produced parity streams, 
as depicted in Fig. 12A. However, alternatively they can be generated from scratch 
5 by assigning a prescribed number of parity packets associated with a particular 
source layer block to the same parity stream. It is noted that the foregoing 
processes associated with the generation of parity streams can be accomplished 
ahead of "ime and the parity streams stored until the time of the multicast. However, 
alternatively, the parity streams can be produced and multicast as each block of the 
10 source layer is multicast, thus resulting in a real-time generation of the parity streams 
and no need to store them. 

Referring to Fig 12B, the next step 1214 in the process is to assign each 
parity stream to one of W waves. In the context of creating the aforementioned 

15 waves, the term parity stream will be used in a broad sense to includes both 
collapsed and uncollapsed streams. Thus, a wave can be made up of all 
uncollapsed streams, all collapsed streams, or a combination of both. In step 1216, 
all the parity streams in the first wave are multicast Without any delay in comparison 
to the source layer. The parity streams in subsequent waves are also multicast, 

20 except that each is delayed in comparison to the parity streams in the wave 

immediately preceding it (step 1218). The delay times are chosen as described in 
connection with the pure pseudo-ARQ error correction technique. 

Referring to Fig. 13A, the receiver's part of the process begins in step 1300 
25 with subscribing to a source layer. In addition, the receiver subscribes to one or 

more parity streams in the first wave associated with the subscribed to source layer, 
or to none at all (step 1302). The next step 1304 in the hybrid process involves 
receiving source layer packets for the incoming data block of the subscribed to 
source layer. If at least one parity stream was subscribed to in step 1302, then in 
30 step 1 306, parity packets associated with the incoming data block are received in the 
subscribed to parity stream(s). However, in the case where no parity streams from 
the first wave are subscribed to, step 1306 would not apply. This latter scenario is 
designated in Fig. 13A by the dashed line arrow by-passing step 1306. 
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Referring now to Fig. 1 3B, it is next determined whether the total number of 
the source layer and parity packets received is at least equal to the number of 
source packets transmitted for the block, as indicated in step 1308. This could be 
accomplished as before by employing the sequence numbers these packets typically 
5 have embedded within the data making up the packet. If it is determined that the 
total number of source layer and parity stream packets received for the incoming 
block equal or exceed the number of source packets transmitted for that block, then 
all the missing source layer packets in the block are recovered using the received 
parity packets (step 1320). This is accomplished by solving Equation (1) for the 

10 particular FEC code employed The missing source layer packets associated with 
the incoming data block are replaced with the aforementioned recovered packets in 
step 1322. If, however, it is determined that the total number of source layer and 
parity stream packets received for the incoming block do not equal or exceed the 
number of source packets transmitted for that block, the next step in the hybrid 

15 process is to determine whether there are any more parity stream waves available 
for subscription (step 1310). If there are parity streams available, in step 1312 one 
or more parity streams in the next wave (i.e., the wave having parity packets that are 
delayed less than any other previously unselected wave) are subscribed to, as 
desired. An attempt is then made in step 1314 to capture parity packet(s) associated 

20 with the incoming data block from the newly subscribed to parity stream(s). Once 
the foregoing parity package capture procedure is complete, the receiver 
unsubscribes from the parity streams(s) in step 1316, and the process of steps 1308 
through 1316 is repeated (or steps 1308 and 1320/1322 in the case where sufficient 
parity packages have.been captured). If, on the other hand, it is determined that 

25 there are no more parity stream waves available for subscription, then the incoming 
block is sent on for further process without recovering any of its missing packets 
(step 1318). 

Finally, as indicated in step 1324, the foregoing recovery process of steps 
30 1304 through 1322 is repeated for each successive incoming data block until the 
multicast ends. Preferably, this is accomplished as each block arrives so as to 
minimize the delay in reconstructing the video and/or audio signal from the source 
layers and to minimize the amount of data that needs to be stored. 

34 



WO 00/76113 



CLAIMS 



PCT/USOO/14060 



What is claimed is: 

5 1 . A computer-implemented error correction process for use in a receiver- 

driven layered multicast of real-time media over a heterogeneous packet network to 
a plurality of receivers, wherein the real-time media is transmitted over the network in 
multiple streams of packetized source data forming hierarchical layers of information, 
said error correction process comprising using a computer to perform the following 
10 step: 

multicasting at least one stream of packetized error correction 
information for at least one of the streams of packetized source data, wherein each 
stream of packetized error correction information is capable of being used by a 
receiver to assist in the recovery of packets of source data lost during transmission. 

15 

2. The process of Claim 1 wherein the step of multicasting streams of 
packetized error correction information comprises the steps of: 

associating each stream of packetized error correction information with 
a one of the multiple streams of packetized source data; and 
20 incorporating in each error correction stream information necessary to 

assist a receiver in recovering said packets of lost source data belonging to the 
particular stream of packetized source data associated with the error correction 
stream. 

25 3 The process of Claim 2, wherein there are multiple streams of error 

correction information associated with each stream of source data, and wherein the 
incorporating step comprises the step of incorporating error correction information in 
each error correction stream associated with the same source data stream which 
makes the streams redundant, said redundant streams making it possible for a 

30 receiver to employ more than one error correction stream and obtain a desired 
amount of the error correction information needed to replace packets in an 
associated source data stream that were lost during the transmission even if some of 
the error correction stream packets themselves are lost in transmission. 
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4. The process of Claim 3, wherein the step of multicasting streams of 
packetized error correction information comprises the step of incorporating an 
identifier into each packet in each of the streams of packetized error correction 
information wherein a different identifier is employed for each of the error correction 

5 information streams, thereby allowing each receiver to select the number of error 
correction information streams to be received by specifying that only packets having 
particular identifiers be routed over the network to the receiver, 

5. The process of Claim 4, further comprising the step of a receiver 

10 improving the quality of the received broadcast by first selecting a desired number of 
source data streams while leaving enough bandwidth available to also select a 
number of error correction information streams for one or more of the source 
streams that will compensate, at least partially, for an inherent packet loss rate 
associated with the receiver's connection to the network, and then selecting said 

15 number of error correction information streams. 

6. A computer-implemented process for multicasting real-time media over 
a heterogeneous packet network to a plurality of receivers, comprising using a 
computer to perform the following steps: 

20 forming a series of source data streams from a media signal, said 

source data streams comprising a base layer representing the media signal at a 
least acceptable level of quality, and at least one enhancement layer which when 
combined with the base layer improves the level of quality of the media signal 
derivable therefrom; - 

25 for each source data stream, creating at ieast one error correction 

information stream from the source layer, each error correction stream comprising a 
series of data packets that can be used by a receiver to assist in the recovery of data 
packets of the associated source layer that may be lost during transmission; 

respectively multicasting each source layer and error correction layer to 

30 a different network address for routing on to a receiver. 

7. A computer-implemented process for a receiver to receive a real-time 
media multicast over a heterogeneous packet network wherein the real-time media is 
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transmitted over the network in multiple streams of packetized source data forming 
hierarchical layers of information and associated error correction streams, 
comprising using a computer to perform the following steps: 

determining a currently available bandwidth associated with the 
5 receiver's connection to the network; 

determining a current packet loss rate associated with receiving data 
over the network; 

selecting at least one source data stream and at least one associated 

error correction information stream which are to be received so as to allow the best 
10 possible level of quality in a media signal reconstructed from received source data 

streams, wherein said selecting is based on the currently available bandwidth and 

the current packet loss rate; 

subscribing over the network to the selected source data streams and 

error correction information streams, and receiving the same; 
15 using the received error correction information stream or streams 

associated with each received source data stream to assist in the recovery of 

packets of that source data stream which were lost during transmission to produce a 

recovered source data stream; and 

reconstruct the media signal from the recovered source data stream or 
20 streams 

8. The process of Claim 7, further comprising the step of continuously 
monitoring the available bandwidth and packet loss rate and repeating the selection 
and subscription steps as needed to maintain the best level of quality of the 

25 reconstructed media signal as possible. 

9. The process of Claim 8, wherein the monitoring step comprises the 
steps of: 

rechecking the available bandwidth and packet loss rate periodically 
30 during the multicast of the real-time media; 

determining whether the available bandwidth has changed more than a 
prescribed maximum bandwidth change threshold; 
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determining whether the packet loss rate has changed more than a 
prescribed maximum packet loss rate change threshold; and 

repeating the selection and subscription steps whenever either or both 
of the prescribed thresholds is exceeded. 

10. A computer-implemented error correction system for use in a rece.ver- 
driven layered multicast of real-time media over a heterogeneous packet network to 
a plurality of receivers, wherein the real-time media is transmitted over the network in 
mult.ple areams of packetized source data forming hierarchical layers of information, 
said error correction system comprising: 

a broadcaster capable of multicasting at least one stream of packetized 
error correction information for at least one of the streams of packetized source data- 
and 

at least one receiver, each of which is capable of using a stream of 
packetized error correction information associated with a particular source data 
stream to assist in the recovery of packets of that source data lost during 
transmission. 



\ 



11. The system of Claim 1 0, wherein the broadcaster comprises: 

an encoder and packetizer module capable of producing said multiple 
stream of packetized source data form a media signal; and 

at least one error correction module, wherein each error correction 
module is capable producing at least one stream of packetized error correction 
information from a stream of packetized source data and thereafter multicasting each 
source data stream and its associated error correction streams onto the network. 

12. The system of Claim 10, wherein each receiver comprises: 

a subscription module capable of instructing the network to route 
particular ones of the source data streams and error correction streams to the 
receiver; 

at least one recovery module capable of receiving a source data 
stream and its associated error correction stream or streams, and thereafter using 
each error correction stream received to assist in the recovery of packets identical to 
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those of the source data stream that were lost during transmission and inserting the 
recovered packets into the source data stream to produce a recovered source data 
stream; and 

a unpacketizer and decoder module for reconstructing a media signal 
5 from the recovered source data stream or streams. 

13. The system of Claim 12, wherein each receiver further comprises a 
subscription optimization module capable of instructing the subscription module 
which source data streams and which error correction streams to request from the 

10 network, wherein the subscription optimization module instructs the subscription 

module based on the number of source data streams it is desired to receive so as to 
obtain a cesired quality in the media signal, while leaving enough bandwidth 
available to also select a number of error correction information streams for one or 
more of the source streams that will compensate, at least partially, for an inherent 

15 packet loss rate associated with the receiver's connection to the network. 

14. An error correction system for use in a receiver-driven layered 
multicast of real-time media over a heterogeneous packet network to a plurality of 
receivers, wherein the real-time media is transmitted over the network in multiple 

20 streams of packetized source data forming hierarchical layers of information, said 
error correction process comprising: 

a general purpose computing device; 

a computer program comprising program modules executable by the 
computing device, wherein the computing device is directed by the program modules 
25 of the cor lputer program to, 

multicast at least one stream of packetized error correction 
information for at least one of the streams of packetized source data, wherein each 
stream of packetized error correction information is capable of being used by a 
receiver to assist in the recovery of packets of source data lost during transmission. 
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15. The system of Claim 14 wherein the program module for multicasting 
streams of packetized error correction information comprises sub-modules for: 
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associating each stream of packetized error correction information with 
a one of tie multiple streams of packetized source data; and 

incorporating in each error correction stream information necessary to 
assist a receiver in recovering said packets of lost source data belonging to the 
particular stream of packetized source data associated with the error correction 
stream. 



16. The system of Claim 1 5, wherein there are multiple streams of error 
correction information associated with each stream of source data, and wherein the 
incorporating sub-module comprises a sub-module for incorporating error correction 
information in each error correction stream associated with the same source data 
stream which makes the streams redundant, said redundant streams making it 
possible for a receiver to employ more than one error correction stream and obtain a 
desired amount of the error correction information needed to replace packets in an 
associated source data stream that were lost during the transmission even if some of 
the error correction stream packets themselves are lost in transmission. 

17. The system of Claim 1 6, wherein the program module for multicasting 
streams of packetized error correction information comprises a sub-module for 
incorporating an identifier into each packet in each of the streams of packetized error 
correction information wherein a different identifier is employed for each of the error 
correction information streams, thereby allowing each receiver to select the number 
of error correction information streams to be received by specifying that only packets 
having particular identifiers be routed over the network to the receiver. 

18. The system of Claim 17, further comprising a program module for a 
receiver improving the quality of the received broadcast by first selecting a desired 
number o, source data streams while leaving enough bandwidth available to also 
select a number of error correction information streams for one or more of the source 
streams that will compensate, at least partially, for an inherent packet loss rate 
associated with the receiver's connection to the network, and then selecting said 
number of error correction information streams. 
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19. An error correction system for multicasting real-time media over a 
heterogeneous packet network to a plurality of receivers, comprising: 

a general purpose computing device; 

a computer program comprising program modules executable by the 
5 computing device, wherein the computing device is directed by the program modules 
of the computer program to, 

form a series of source data streams from a media signal, said 
source data streams comprising a base layer representing the media signal at a 
least acceptable level of quality, and at least one enhancement layer which when 
10 combined with the base layer improves the level of quality of the media signal 
derivable therefrom. 

for each source data stream, create at least one error correction 
information stream from the source layer, each error correction stream comprising a 
series of data packets that can be used by a receiver to assist in the recovery of data 
15 packets of the associated source layer that may be lost during transmission, and 

respectively multicast each source layer and error correction 
layer to a different network address for routing on to a receiver. 

20. An error correction system for allowing a receiver to receive a real-time 
20 media multicast over a heterogeneous packet network wherein the real-time media is 

transmitted over the network in multiple streams of packetized source data forming 
hierarchical layers of information and associated error correction streams, 
comprising: 

a general purpose computing device; 
25 a computer program comprising program modules executable by the 

computing device, wherein the computing device is directed by the program modules 
of the computer program to, 

determine a currently available bandwidth associated with the 
receiver's connection to the network, 
30 determine a current packet loss rate associated with receiving 

data over the network, 

select at least one source data stream and at least one 
associated error correction information stream which are to be received so as to 
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a computer-readable storage medium; and 

a computer program comprising program modules stored in the storage 
medium, wherein the storage medium is so configured by the computer program that 
it causes the computer to, 

5 multicast at least one stream of packetized error correction 

information for at least one of the streams of packetized source data, wherein each 
stream of packetized error correction information is capable of being used by a 
receiver to assist in the recovery of packets of source data lost during transmission. 

10 24 - The computer-readable memory of Claim 23 wherein the program 

module for multicasting streams of packetized error correction information comprises 
sub-modules for: 

associating eac:n stream of packetized error correction information with 
a one of the multiple streams of packetized source data; and 
15 incorporating in each error correction stream information necessary to 

assist a receiver in recovering said packets of lost source data belonging to the 
particular stream of packetized source data associated with the error correction 
stream. \ 

20 25. The computer-readable memory of Claim 24, wherein there are 

multiple streams of error correction information associated with each stream of 
source data, and wherein the incorporating sub-module comprises a sub-module for 
incorporating error correction information in each error correction stream associated 
with the same source„data stream which makes the streams redundant, said 

25 redundant streams making it possible for a receiver to employ more than one error 
correction stream and obtain a desired amount of the error correction information 
needed to replace packets in an associated source data stream that were lost during 
the transmission even if some of the error correction stream packets themselves are 
lost in transmission. 



30 



26. The computer-readable memory of Claim 25, wherein the program 
module for multicasting streams of packetized error correction information comprises 
a sub-module for incorporating an identifier into each packet in each of the streams 
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of packetized. error correction information wherein a different identifier is employed 
for each of the error correction information streams, thereby allowing each receiver 
to select the number of error correction information streams to be received by 
specifying that only packets having particular identifiers be routed over the network 
5 to the receiver. 

27. The computer-readable memory of Claim 26, further comprising a 
program module for a receiver improving the quality of the received broadcast by first 
selecting a desired number of source data streams while leaving enough bandwidth 
10 available to also select a number of error correction information streams for one or 
more of the source streams that will compensate, at least partially, for an inherent 
packet loss rate associated with the receiver's connection to the network, and then 
selecting said number of error correction information streams. 

15 28. A computer-readable memory for multicasting real-time media over a 

heterogeneous packet netwo. k to a plurality of receivers, comprising: 
a computer-readable storage medium; and 

a computer program comprising program modules stored in the storage 
medium, wherein the storage medium is so configured by the computer program that 
20 it causes the computer to, 

form a series of source data streams from a media signal, said 
source data streams comprising a base layer representing the media signal at a 
least acceptable level of quality, and at least one enhancement layer which when 
combined with the base layer improves the level of quality of the media signal 
25 derivable therefrom. 

for each source data stream, create at least one error correction 
information stream from the source layer, each error correction stream comprising a 
series of data packets that can be used by a receiver to assist in the recovery of data 
packets of the associated source layer that may be lost during transmission, and 
30 respectively multicast each source layer and error correction 

layer to a different network address for routing on to a receiver. 
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29. A computer-readable memory for allowing a receiver to receive a real- 
time media multicast over a heterogeneous packet network wherein the real-time 
media is transmitted over the network in multiple streams of packetized source data 
forming hierarchical layers of information and associated error correction streams, 

5 comprising: 

a computer-readable storage medium; and 

a computer program comprising program modules stored in the storage 
medium, wherein the storage medium is so configured by the computer program that 
it causes the computer to, 

10 determine a currently available bandwidth associated with the 

receiver's connection to the network, 

determine a current packet loss rate associated with receiving 
data over the network, 

select at least one source data stream and at least one 
15 associated error correction information stream which are to be received so as to 

allow the Dest possible level of quality in a media signal reconstructed from received 
source data streams, wherein said selecting is based on the currently available 
bandwidth and the current packet loss rate, \ 

subscribe over the network to the selected source data streams 
20 and error correction information streams, and receiving the same, 

use the received error correction information stream or streams 
associated with each received source data stream to assist in the recovery of 
packets of that source data stream which were lost during transmission to produce a 
recovered source data stream, and 
25 reconstruct the media signal from the recovered source data 

stream or streams. 

30. The computer-readable memory of Claim 29, further comprising a 
program module for continuously monitoring the available bandwidth and packet loss 

30 rate and repeating the selection and subscription steps as needed to maintain the 
best level of quality of the reconstructed media signal as possible. 
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31 . The computer-readable memory of Claim 30, wherein the monitoring 
sub-module comprises sub-modules for: 

rechecking the available bandwidth and packet loss rate periodically 
during the multicast of the real-time media; 

determining whether the available bandwidth has changed more than a 
prescribed maximum bandwidth change threshold; 

determining whether the packet loss rate has changed more than a 
prescribed maximum packet loss rate change threshold; and 

repeating the selection and subscription steps whenever either or both 
of the prescribed thresholds is exceeded. 

32. A computer-implemented error correction process for use in a receiver- 
driven layered multicast of real-time media over a heterogeneous packet network to 
a plurality of receivers, wherein the real-time media is transmitted over the network in 
multiple streams of packetized source data forming hierarchical layers of information, 
said error correction process comprising using a computer to perform the following 
steps: 

producing at least one stream of packetized error correction 
information for at least one of the streams of packetized source data, wherein each 
error correction stream comprises parity packets encoded from packets of the source 
data which have been subjected to a linear transform, said linear transform 
producing a series of parity packets which can be employed by a receiver to assist in 
the recovery of lost source data packets using a reverse transform; 

multicasting each stream of packetized error correction information to 
said receivers to allow each receiver to recover packets of source data lost during 
transmission. 

33. The process of Claim 32, wherein the step of encoding the error 
correction packets comprises the step of applying a Forward Error Correction (FEC) 
encoding technique to each source data stream. 

34. The process of Claim 32 wherein the step of multicasting streams of 
packetized error correction information comprises the steps of: 
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associating each stream of packetized error correction information with 
a one of the multiple streams of packetized source data; and 

incorporating in each error correction stream information necessary to 
assist a receiver in recovering said packets of lost source data belonging to the 
5 particular stream of packetized source data associated with the error correction 
stream. 

35 The process of Claim 34, wherein there are multiple streams of error 
correction information associated with each stream of source data, and wherein the 

10 incorporating step comprises the step of incorporating error correction information in 
each error correction stream associated with the same source data stream which 
makes the streams redundant, said redundant streams making it possible for a 
receiver to employ more than one error correction stream and obtain a desired 
amount of the error correction information needed to replace packets in an 

15 associated source data stream that were lost during the transmission even if some of 
the error correction stream packets themselves are lost in transmission. 

36. The process of Claim 35, wherein the step of broadcasting streams of 
packetized error correction information comprises the step of incorporating an 

20 identifier into each packet in each of the streams of packetized error correction 

information wherein a different identifier is employed for each of the error correction 
information streams, thereby allowing each receiver to select the number of error 
correction information streams to be received by specifying that only packets having 
particular identifiers be routed over the network to the receiver. 

25 

37. The process of Claim 36, further comprising~the step of a receiver 
improving the quality of the received broadcast by first selecting a desired number of 
source data streams while leaving enough bandwidth available to also select a 
number of error correction information streams for one or more of the source 

30 streams that will compensate, at least partially, for an inherent packet loss rate 
associated with the receiver's connection to the network, and then selecting said 
number of error correction information streams. 
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38. A computer-implemented process for multicasting real-time media over 
a heterogeneous packet network to a plurality of receivers, comprising using a 
computer to perform the following steps: 

forming a series of source data streams from a media signal, said 
source data streams comprising a base layer representing the media signal at a 
least acceptable level of quality, and at least one enhancement layer which when 
combined with the base layer improves the level of quality of the media signal 
derivable therefrom; 

for each source data stream, creating at least one error correction 
information stream from the source layer, wherein each error correction stream 
comprises parity packets encoded from packets of the source data stream which 
have been subjected to a linear transform, said linear transform producing a series of 
parity packets which can be employed to assist in the recovery of lost source data 
packets using a reverse transform; 

respectively multicasting each source layer and error correction layer to 
a different network address for routing on to a receiver; 

for each receiver, receiving at least one source data stream and at 
least one error correction layer, and reconstructing a x media signal from the at least 
one source data stream after having used the at least one error correction stream to 
recover source data packets lost in transmission of the source data stream. 

39. The process of Claim 38, wherein the step of creating at least one error 
correction information stream from a source layer comprises the steps of: 

(a) partitioning the source data stream into a plurality of data blocks 
each containing k data packets; 

(b) selecting a previously unselected data block; 

(c) applying a Forward Error Correction (FEC) encoding technique 
to the chosen data block to produce n-k parity packets for that block where n > k; 

(d) assigning each parity packet to a different error correction 

stream; and 

(e) repeating steps (b) through (d) for each data block. 
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40. The process of Claim 39 wherein the receiving step comprises the 
steps of: 

subscribing over the network to at least one source data stream and at 
least one error correction information stream; 
5 for each subscribed to source data stream, 

(i) receiving source data stream packets comprising a next 
incoming source data block of the subscribed to source data stream; 

(ii) receiving parity packets associated with the incoming 
source data block in the subscribed to error correction information stream or 

10 streams; 

\ (iii) determining whether the total number of source data 

stream packets and parity packets received for the incoming source data block is at 
least equal the number of source data stream packets multicast for the block; 

(iv) whenever the total number of source diata stream packets 
15 and parity packets received for the incoming source data block equals or exceeds 

the number of source data stream packets multicast for the block, recovering all the 
source data stream packets missing in the incoming source data block using the 
received parity packets, 

(v) replacing said missing source data stream packets using 
• 20 the recovered packets, and 

(vi) repeating steps (i) through (v) for each successive 
incoming source data block. 

41 . The process of Claim 40, wherein the subscribing step comprises the 
25 steps of: 

determining a currently available bandwidth associated with the 
receiver's connection to the network; 

determining a current packet loss rate associated with receiving data 
over the network; 

30 selecting at least one source data stream and at least one associated 

error correction stream which are to be received so as to allow the best possible level 
of quality in a media signal reconstructed from received source data streams, wherein 
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said selecting is based on the currently available bandwidth and the current packet 
loss rate; and 

subscribing to each selected source data stream and each selected 
error correction information stream. 

5 

42. An error correction system for use in a receiver-driven layered 
multicast of real-time media ever a heterogeneous packet network to a plurality of 
receivers, wherein the real-time media is transmitted over the network in multiple 
streams of packetized source data forming hierarchical layers of information, said 
10 error correction system comprising: 

a general purpose computing device; 

a computer program comprising program modules executable by the 
computing device, wherein the computing device is directed by the program modules 
of the computer program to, 

15 produce at least one stream of packetized error correction 

information for at least one of the streams of packetized source data, wherein each 
error correction stream comprises parity packets encoded from packets of the source 
data which have been subjected to a linear transform; said linear transform 
producing a series of parity packets which can be employed by a receiver to assist in 

20 the recovery of lost source dota packets using a reverse transform, 

multicast each stream of packetized error correction information 
to said receivers to allow each receiver to recover packets of source data lost during 
transmission. 

25 43 Tn e system of Claim 42, wherein the program module for encoding the 

error correction packets comprises a sub-module for applying a Forward Error 
Correction (FEC) encoding technique to each source data stream. 

44. The system of Claim 42 wherein the program module for multicasting 
30 streams of packetized error correction information comprises sub-modules for: 

associating each stream of packetized error correction information with 
a one of the multiple streams of packetized source data; and 
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incorporating in each error correction stream information necessary to 
assist a receiver in recovering said packets of lost source data belonging to the 
particular stream of packetized source data associated with the error correction 
stream. 

5 

45. The system of Claim 44, wherein there are multiple streams of error 
correction information associated with each stream of source data, and wherein the 
incorporating sub-module comprises a sub-module for incorporating error correction 
information in each error correction stream associated with the same source data 

10 stream which makes the streams redundant, said redundant streams making it 

possible for a receiver to employ more than one error correction stream and obtain a 
desired amount of the error correction information needed to replace packets in an 
associated source data stream that were lost during the transmission even if some of 
the error correction stream packets themselves are lost in transmission. 

15 

46. The system of Claim 45, wherein the program module for multicasting 
streams of packetized error correction information comprises a sub-module for 
incorporating an identifier into each packet in each df the streams of packetized error 
correction information wherein a different identifier is employed for each of the error 

20 correction information streams, thereby allowing each receiver to select the number 
of error correction information streams to be received by specifying that only packets 
having particular identifiers be routed over the network to the receiver. 

47. The system of Claim 46, further comprising a program module for 

25 allowing a receiver to improve the quality of the received broadcast by first selecting 
a desired number of source data streams while leaving enough bandwidth available 
to also select a number of error correction information streams for one or more of the 
source streams that will compensate, at least partially, for an inherent packet loss 
rate associated with the receiver's connection to the network, and then selecting said 

30 number of error correction information streams. 

48 An error correction system for multicasting real-time media over a 
heterogeneous packet network to a plurality of receivers, comprising: 
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a general purpose computing device; 

a computer program comprising program modules executable by the 
computing device, wherein the computing device is directed by the program modules 
of the computer program to, 

form a series of source data streams from a media signal, said 
source data streams comprising a base layer representing the media signal at a 
least acceptable level of quality, and at least one enhancement layer which when 
combined with the base layer improves the level of quality of the media signal 
derivable therefrom, 

for each source data stream, create at least one error correction 
information stream from the source layer, wherein each error correction stream 
comprises parity packets encoded from packets of the source data stream which 
have been subjected to a linear transform, said linear transform producing a series of 
parity packets which can be employed to assist in the recovery of lost source data 
packets using a reverse transform, 

respectively multicasting each source layer and error correction 
layer to a different network address for routing on to a receiver; and 

for each receiver, receiving at least one source data stream and at least one 
error correction layer, and reconstructing a media signal from the at least one source 
data stream after having used the at least one error correction stream to recover 
source data packets lost in transmission of the source data stream. 

49. The system of Claim 48, wherein the program module for creating at 
least one error correction information stream from a source layer comprises sub- 
modules for: 

(a) partitioning the source data stream into a plurality of data blocks 
each containing k data packets; 

(b) selecting a previously unselected data block; 

(c) applying a Forward Error Correction (FEC) encoding technique 
to the chosen data block to produce n-k parity packets for that block where n > k; 

(d) assigning each parity packet to a different error correction 

stream; and 

(e) repeating steps (b) through (d) for each data block. 
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50. The system of Claim 49 wherein the receiving program module 
comprises sub-modules for: 

subscribing over the network to at least one source data stream and at 
least one error correction information stream; 
5 for each subscribed to source data stream, 

(i) receiving source data stream packets comprising a next 
incoming source data block of the subscribed to source data stream; 

(ii) receiving parity packets associated with the incoming 
source data block in the subscribed to error correction information stream or 

10 streams; 

(iii) determining whether the total number of source data 
stream packets and parity packets received for the incoming source data block is at 
least equal the number of source data stream packets multicast for the block; 

(iv) whenever the total number of source data stream packets 
15 and parity packets received for the incoming source data block equals or exceeds 

the number of source data stream packets multicast for the block, recovering all the 
source data stream packets missing in the incoming source data block using the 
received parity packets, \ 

(v) replacing said missing source data stream packets using 
20 the recovered packets, and 

(vi) repeating steps (i) through (v) for each successive 
incoming source data block. 

51 . The system of Claim 50, wherein the subscribing sub-module 
25 comprises sub-modules for: 

determining a currently available bandwidth associated with the 
receivers connection to the network; 

determining a current packet loss rate associated with receiving data 
over the network; 

30 selecting at least one source data stream and at least one associated 

error correction stream which are to be received so as to allow the best possible level 
of quality in a media signal reconstructed from received source data streams, wherein - 
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said selecting is based on the currently available bandwidth and the current packet 
loss rate; rnd 

subscribing to each selected source data stream and each selected 
error correction information stream. 

5 

52. A computer-readable memory for use in a receiver-driven layered 
multicast of real-time media over a heterogeneous packet network to a plurality of 
receivers, wherein the real-time media is transmitted over the network in multiple 
streams of packetized source data forming hierarchical layers of information, 
10 comprising: 

a computer-readable storage medium; and 

a computer program comprising program modules stored in the storage 
medium, wherein the storage medium is so configured by the computer program that 
it causes the computer to, 

15 produce at least one stream of packetized error correction 

information for at least one of the streams of packetized source data, wherein each 
error correction stream comprises parity packets encoded from packets of the source 
data which have been subjected to a linear transform, said linear transform 
producing a series of parity packets which can be employed by a receiver to assist in 

20 the recovery of lost source data packets using a reverse transform, 

multicast each stream of packetized error correction information 
to said receivers to allow each receiver to recover packets of source data lost during 
transmission. 

25 53, The computer-readable memory of Claim 52, wherein the program 

module for encoding the error correction packets comprises a sub-module for 
applying a Forward Error Correction (FEC) encoding technique to each source data 
stream. 

30 54. The computer-readable memory of Claim 52 wherein the program 

module for multicasting streams of packetized error correction information comprises 
sub-modules for: 
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associating each stream of piacketized error correction information with 
a one of the multiple streams of packetized source data; and 

incorporating in each error correction stream information necessary to 
assist a receiver in recovering said packets of lost source data belonging to the 
5 particular stream of packetized source data associated with the error correction 
stream. 



55. The computer-readable memory of Claim 54, wherein there are 
multiple streams of error correction information associated with each stream of 

10 source data, and wherein the incorporating sub-module comprises a sub-module for 
incorporating error correction information in each error correction stream associated 
with the same source data stream which makes the streams redundant, said 
redundant streams making it possible for a receiver to employ more than one error 
correction stream and obtain a desired amount of the error correction information 

15 needed to replace packets in an associated source data stream that were lost during 
the transmission even if some of the error correction stream packets themselves are 
lost in transmission. 

56. The computer-readable memory of Claim 55, wherein the program 

20 module for multicasting streams of packetized error correction information comprises 
a sub-module for incorporating an identifier into each packet in each of the streams 
of packetized error correction information wherein a different identifier is employed 
for each of the error correction information streams, thereby allowing each receiver 
to select the number of error correction information streams to be received by 

25 specifying that only packets having particular identifiers be routed over the network 
to the receiver. 



57. The computer-readable memory of Claim 56, further comprising a 
program module for allowing a receiver to improve the quality of the received 
30 broadcast by first selecting a desired number of source data streams while leaving 
enough bandwidth available to also select a number of error correction information 
streams for one or more of the source streams that will compensate, at least 
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partially, for an inherent packet loss rate associated witt, the received connection to 
the network, and then selecting said number of error correction information streams. 



58. 



A computer-readable memory for multicasting real-time media over a 
heterogeneous packet network to a plurality of receivers, comprising: 
a computer-readable storage medium; and 

a computer program comprising program modules stored in the storage 
m-um. wherein the storage medium is so configured by the computer program that 
it causes the computer to, 

form a series of source data streams from a media signal said 
source data streams comprismg a base layer representing the media signal a. a 
leas, acceptable level of quality, and a, leas, one enhancement layer which when 
combmed with ,he base iayer improves the level of quality of the media signal 
denvable therefrom, 

for each source data stream, create at least one error correction 
-formation stream from the source layer, wherein each enor comaction stream 
comprises parity packets encoded from packets of the source data stream which 
have been subjected to a .inear (ransfonn, said linear transform producing a series of 
panty packets which can be employed ,o assist in the recovery of lost source date 
packets using a reverse transform, 

respectively multicasting each source layer and error correction 
layer to a different network address for routing on to a receiver; and 

for each receiver, receiving at least one source data stream and at least one 
error correction .ayer, and reconstructing a media signa, from the at .east one source 
data stream after having used the at least one error correction stream to recover 
source data packets lost in transmission of the source data stream. 

59. The computer-readable memory of Claim 58, wherein the program 
module for creating at .east one error correction information stream from a source 
layer comprises sub-modules for: 

(a) partitioning the source data stream into a plurality of data blocks 
each containing k data packets; 

(b) selecting a previously unselected data block; 
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(c) applying a Forward Error Correction (FEC) encoding technique 
to the chosen data block to produce n-k parity packets for that block where n > k; 

(d) assigning each parity packet to a different error correction 

stream; and 

5 (e) repeating steps (b) through (d) for each data block. 

60. The computer-readable memory of Claim 59 wherein the receiving 
program module comprises sub-modules for: 

subscribing over the network to at least one source data stream and at 
10 least one error correction information stream; 

for each subscribed to source data stream, 

(i) receiving source data stream packets comprising a next 
incoming source data block of the subscribed to source data stream; 

(ii) receiving parity packets associated with the incoming 
15 source data block in the subscribed to error correction information stream or 

streams; 

(iii) determining whether the total number of source data 
stream packets and parity packets received for the irtcoming source data block is at 
least equal the number of source data stream packets multicast for the block; 

20 (iv) whenever the total number of source data stream packets 

and parity packets received for the incoming source data block equals or exceeds 
the number of source data stream packets multicast for the block, recovering all the 
source data stream packets missing in the incoming source data block using the 
received parity packets, 

25 (v) replacing said missing source data stream packets using 

the recovered packets, and 

(vi) repeating steps (i) through (v) for each successive 
incoming source data block. 

30 61 . The computer-readable memory of Claim 60, wherein the subscribing 

sub-module comprises sub-modules for: 

determining a currently available bandwidth associated with the 
receivers connection to the network; 

57 



OCID: <WO__00761 13A1_I_> 



WO 00/761 13 PCT/US00/14060 

determining a current packet loss rate associated with receiving data 
over the network; 

selecting at least one source data stream and at least one associated 
error correction stream which are to be received so as to allow the best possible level 
of quality in a media signal reconstructed from received source data streams, wherein 
said selecting is based on the currently available bandwidth and the current packet 
loss rate; and 

subscribing to each selected source data stream and each selected 
error correction information stream. 

62. A computer-implemented error correction process for use in a receiver- 
driven layered multicast of real-time media over a heterogeneous packet network to 
a plurality of receivers, wherein the real-time media is transmitted over the network in 
multiple streams of packetized source data forming hierarchical layers of information, 
said error correction process comprising using a computer to perform the following 
steps: 

producing at least one stream of packetized error correction 
information for at least one of the streams of packetfred source data, wherein each 
error correction stream comprises a delayed version of the stream of packetized 
source data associated therewith; 

broadcasting the streams of packetized error correction information to 
said receivers to allow each receiver to replace packets of source data lost during 
transmission. 

63. The process of Claim 62 wherein the step of broadcasting streams of 
packetized error correction information comprises the steps of: 

associating each stream of packetized error correction information with 
a one of the multiple streams of packetized source data; and 

incorporating in each error correction stream information necessary to 
assist a receiver in recovering said packets of lost source data belonging to the 
particular stream of packetized source data associated with the error correction 
stream. 
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64. The process of Claim 63, wherein there are multiple streams of error 
correction information associated with each stream of source data, and wherein the 
incorporating step comprises the step of incorporating error correction information in 
each error correction stream associated with the same source data stream which 
makes the streams redundant, said redundant streams making it possible for a 
receiver to employ more than one error correction stream and obtain a desired 
amount of the error correction information needed to replace packets in an 
associated source data stream that were lost during the transmission even if some of 
the error correction stream packets themselves are lost in transmission. 



65. The process of Claim 64, wherein the step of broadcasting streams of 
packetized error correction information comprises the step of incorporating an 
identifier into each packet in each of the streams of packetized error correction 
information wherein a different identifier is employed for each of the error correction 

15 information streams, thereby allowing each receiver to select the number of error 
correction information streams to be received by specifying that only packets having 
particular identifiers be routed over the network to the receiver. 

66. The process of Claim 65, further comprising the step of a receiver 

20 improving the quality of the received broadcast by first selecting a desired number of 
source data streams while leaving enough bandwidth available to also select a 
number of error correction information streams for one or more of the source 
streams that will compensate, at least partially, for an inherent packet loss rate 
associated with the receiver's connection to the network, and then selecting said 

25 number of error correction information streams. 

67. A computer-implemented process for multicasting real-time media over 
a heterogeneous packet network to a plurality of receivers, comprising using a 
computer to perform the following steps: 

30 forming a series of source data streams from a media signal, said 

source data streams comprising a base layer representing the media signal at a 
least acceptable level of quality, and at least one enhancement layer which when 
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combined with the base layer improves the teve, of quality of the media signal 
derivable therefrom; 

for eaoh souroe data stream, oreating at least one error correction 
■nformanon stream from the source ,a ye , wberein each error correction stream 
compnses a deiayed version o, the stream of source data associated therewith and 
each erro- correction stream associated with the same source data stream is 
progressively delayed from one another; 

a diffe , h reSPeC,iVe ' y m ~"9 «oh source iayer and error correction layer to 
a d,fferent network address for routing on to a receiver 

for each receiver, receiving at least one source data stream and 
penod.cally receiving a, leas, one error correction stream, and reconstruct a 

one error correction stream fo recover source data packets ios. in transmission of the 
source data stream. 

steps of 8 ^ PrOCeSS ° f C,ajm ^ Wh6rein ^ r6CeiVing StSp C ° mprises the 

subscribing over the network to at least one source data stream; 
for each subscribed to source data stream, 

data packet- & the ,ncomin 9 sour <* data stream for a missing 

to th. . t h i 00 Wh6neVer 8 m '' SSing d3ta PaCk6t is detected ' subscribing 
to the least de.ayed version of the error correction streams associated with the 
source data stream, v 

(iii) attempting to recover the missing source data stream 
packet from the subscribed to error correction stream, 

(iv) unsubscribing from the subscribed to error correction 

stream, 

(V) re P lacin 9 *e missing source data stream packet using 
he recovered packet whenever the missing packet is recovered from the subscribed 
to error correction stream, 

(vi) repeating steps (i) through (v) for the duration of the 

multicast. 
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69. The process of Claim 67, wherein said receiving step comprises, the 
steps of: 

subscribing over the network to at least one source data stream 
for each subscribed to source data stream, 
5 (i) monitoring the incoming source data stream for a missing 

data packet; 

(ii) whenever a missing data packet is detected, subscribing 
to the least delayed version of the error correction streams associated with the 
source data stream available on the network, 
1 0 (iii) attempting to recover the missing source data stream 

packet from the subscribed to error correction stream, 

(iv) unsubscribing from the currently subscribed to error 

correction stream, 

(v) whenever the missing packet cannot be recovered from 
15 the previously subscribed to error correction stream, determining whether there is at 

least one more error correction stream associated with the source data stream 
available on the network for subscription 

(vi) whenever the is another error correction stream available, 
subscribing to the least delayed version of the error correction streams associated 

20 with the source data stream available on the network, which has not been 
subscribed to before in connection with said missing data packet, 

(vii) repeating steps (iii) through (vi) until the missing packet is 
recovered or it is determined there are no more previously unsubscribed to error 
correction streams available on the network, 

25 (viii) whenever the missing packet is recovered from a 

subscribed to error correction stream, replacing the missing source data stream 
packet using the recovered packet , 

(vi) repeating steps (i) through (viii) for the duration of the 

multicast. 
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70. The process of Claim 68, wherein the step of subscribing to the least 
delayed version of the error correction streams associated with the source data 
stream comprises the steps of: 
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determining a currently available bandwidth associated with the 
receiver's connection to the network; 

unsubscribing from a subscribed to higher level source data stream to 
subscribe to said least delayed version of the error correction stream whenever there 
is not sufficient bandwidth available to subscribe to both. 

71. The process of Claim 70, further comprising the step of re-subscribing 
to said higher-level source data stream, whenever said least delayed version ot the 
error correction stream is unsubscribed to thereby freeing up the bandwidth needed 
to re-subscribe to the higher-level source data stream. 

72. A computer-implemented process for multicasting real-time media over 
a heterogeneous packet network to a plurality of receivers, comprising using a 
computer to perform the following steps: 

forming a series of source data streams from a media signal, said 
source data streams comprising a base layer representing the media signal at a 
least acceptable level of quality, and at least one enhancement layer which when 
combined with the base layer improves the level of quality of the media signal 
derivable therefrom; 

for each source data stream, creating at least one error correction 
information stream from the source layer, wherein each error correction stream 
comprises parity packets encoded from packets of the source data stream which 
have been subjected to a linear transform, said linear transform producing a series of 
parity packets which can be employed to assist in the recovery of lost source data 
packets using a reverse transform, and wherein at least one of said at least one error 
correction information streams is delayed in comparison to the source data stream; 

respectively multicasting each source layer and error correction layer to 
a different network address for routing on to a receiver; 

for each receiver, receiving at least one of said source data stream and 
receiving, at least periodically, said at least one error correction layer, and 
reconstructing a media signal from the at least one source data stream after having 
used said at least one error correction stream to recover source data packets lost in 
transmission of the source data stream. 



62 



W O 00/761 13 PCT/USOO/14060 

73. The process of Claim 72, wherein there are multiple error correction 
streams associated with each source data stream, and wherein the step of creating 
at least one error correction information stream from the source layer comprises the 
steps of: 

5 (a) partitioning the source data stream into a plurality of data blocks 

each containing k data packets; 

(b) selecting a previously unselected data block; 

(c) applying a Forward Error Correction (FEC) encoding technique 
to the chosen data block to produce n-k parity packets for that block where n > k; 

10 ( d ) assigning each parity packet to a different error correction 

stream; 

(e) repeating steps (b) through (d) for each data block; and 

(f) assigning each error correction stream to one of W waves. 

15 74 The process of Claim 73, wherein the multicasting step comprises the 

steps of: 

multicasting all ;he error correction streams forming a first wave without 
delay in comparison to the source data stream; and v 

multicasting the error correction streams in each subsequent wave with 
20 a delay in comparison to the immediately preceding wave. 

75. The process of Claim 74, wherein said receiving step comprises, the 
steps of: 

subscribing over the network to at least one source data stream; 
25 subscribing to at least one error correction information stream 

associated with the first wave of error correction streams for each subscribed to 
source data stream; 

for each subscribed to source data stream, 

(i) receiving source data stream packets comprising a next 
30 incoming source data block of the subscribed to source data stream; 

(ii) receiving parity packets associated with the incoming 
source data block in the subscribed to error correction information stream or streams 
in the first wave; 
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(iii) determining whether the total number of source data 
stream packets and parity packets received for the incoming source data block is at 
least equal the number of source data stream packets multicast for the block; 

(iv) whenever the total number of source data stream packets, 
and parity packets received for the incoming source data block does not equal or 
exceed the number of source data stream packets multicast for the block, 
determining whether there are any more waves available for subscription on the 
network, 

(v) if there are one or more waves available, subscribing to 
at least one error correction stream in a previously unsubscribed to wave having the 
least delay among the available waves, and attempting to capture the number of 
parity packets associated with the incoming data block which would cause the total 
number of source data stream packets and parity packets received for the incoming 
source data block to equal or exceed the number of source data stream packets 
multicast for the block, and thereafter unsubscribing from said at least one error 
correction stream associated with the delayed wave, 

(vi) repeating step (iii) through (v) until the total number of 
source data stream packets and parity packets received for the incoming source 
data block equals or exceeds the number of source data stream packets multicast 
for the block, or it is determined there are no more waves available, 

(vii) whenever the total number of source data stream packets 
and parity packets received for the incoming source data block equals or exceeds 
the number of source data stream packets multicast for the block, recovering all the 
source data stream packets missing in the incoming source data block using the 
received parity packets, 

(viii) replacing said missing source data stream packets using 
the recovered packets, and 

(ix) repeating steps (i) through (viii) for each successive 
incoming source data block. 

76. The process of Claim 75, further comprising performing, prior to the 
step of assigning each error correction stream to one of W waves, the step of 
collapsing at least some of the error correction streams such that all the parity 
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packets associated with these error correction streams are sequentially included 
within the collapsed error correction stream, and wherein the steps of subscribing to 
any error correction stream include subscribing to a collapsed error correction 
stream, as desired. 

5 

77. The process of Claim 75, wherein the steps of subscribing over the 
network to at least one source data stream and subscribing to at least one error 
correction information stream associated with the first wave of error correction 
streams, comprises the steps of: 

10 determining a currently available bandwidth associated, with the 

receiver's connection to the network; 

determining a current packet loss rate associated with receiving data 
over the network; 

selecting the number of source data streams and the number of 
1 5 associated error correction streams which are to be received so as to allow the best 
possible level of quality in a media signal reconstructed from received source data 
streams, wherein said selecting is based on the currently available bandwidth and the 
current packet loss rate; and \ 

subscribing to each selected source data stream and each selected 
20 error correction information stream. 

78. The process of Claim 75, wherein the step of subscribing to at least 
one error correction stream in a previously unsubscribed to wave, comprises the 
steps of: 

25 determining a currently available bandwidth associated with the 

receiver's connection to the network; 

discontinuing reception of a higher level source data stream to receive 
the at least one error correction stream whenever there is not sufficient bandwidth 
available to receive both. 
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79. The process of Claim 74, wherein said receiving step comprises, the 
steps of: 

subscribing over the network to at least one source data stream; 

65 



<WO 00761 13A1J_> 



WO 00/76113 

PCT/US00/14060 

for each subscribed to source data stream, 

(i) receiving source data stream packets comprising a next 
incoming source data block of the subscribed to source data stream; 

(ii) determining whether any of the source data stream 
packets in the incoming source data block were lost during transmission; 

(iii) whenever it is determined that any source data stream 
packets were lost during transmission of the incoming source data block 
determining whether there are error correction stream avai.ab.e for subscription on 
the network, 

(iv) if there are one or more waves available, subscribing to 
at least one error correction stream in a previously unsubscribed to wave having the 
least delay among the available waves, and attempting to capture the number of 
panty packets associated with the incoming data block which would cause the total 
number of source data stream packets and parity packets received for the incoming 
source data block to equal or exceed the number of source data stream packets 
mult.cast for the block, and thereafter unsubscribing from said at least one error 
correction stream associated with the delayed wave, 

(v) repeating step (iv) until the total number of source data 
stream packets and parity packets received for the incoming source data block 
equals or exceeds the number of source data stream packets multicast for the block 
or there are no more waves available for subscription, 

(vi) whenever the total number of source data stream packets 
and parity packets received for the incoming source data block equals or exceeds 
the number of source.data stream packets multicast for the block, recovering all the 
source data stream packets missing in the incoming source data block using the 
received parity packets, 

(vii) replacing said missing source data stream packets using 
the recovered packets, and 

(ix) repeating steps (i) through (viii) for each successive 
incoming source data block. 

80. The process of Claim 79, further comprising performing, prior to the 
step of assigning each error correction stream to one of W waves, the step of 
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collapsing at least some of the error correction streams such that all the parity 
packets associated with these error correction streams are sequentially included 
within the collapsed error correction stream, and wherein the steps of subscribing to 
any error correction stream include subscribing to a collapsed error correction 
5 stream, as desired. 

81 . The process of Claim 79, wherein the step of subscribing to at least 
one error correction stream in a previously unsubscribed to wave, comprises the 
steps of: 

10 determining a currently available bandwidth associated with the 

receiver's connection to the network; 

discontinuing reception of a higher level source data stream to receive 
the at least one error correction stream whenever there is not sufficient bandwidth 
available to receive both. 

15 

82. An error correction system for use in a receiver-driven layered 
multicast of real-time media over a heterogeneous packet network to a plurality. of 
receivers, wherein the real-time media is transmitted over the network in multiple 
streams of packetized source data forming hierarchical layers of information, said 

20 error correction system comprising: 

a general purpose computing device; 

a computer program comprising program modules executable by the 
computing device, wherein the computing device is directed by the program modules 
of the computer program to, 

25 produce at least one stream of packetized error correction 

information for at least one of the streams of packetized source data, wherein each 
error correction stream comprises a delayed version of the stream of packetized 
source data associated therewith, 

multicast the streams of packetized error correction information 

30 to said receivers to allow each receiver to replace packets of source data lost during 
transmission. 
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83. The system of Claim 82 wherein the program module for multicasting 
streams of packetized error correction information comprises sub-modules for 

assorting each stream of packetized error correction information with 
a one of the multiple streams of packetized source data; and 

incorporating in each error correction stream information necessary to 
as S ,st a receiver in recovering said packets of lost source data belonging to the 
part-cular stream of packetized source data associated with the error correction 
stream. 

84. The system of Claim 83, wherein there are multiple streams of error 
correction information associated with eaoh stream of source data, and wherein the 
■ncorporating sub-module comprises a sub-module for incorporating error correotion 
.nfom^on in each error correction stream associated with the same source date 
stream which makes the streams redundant, said redundant streams making it 
possible for a receiver to employ more than one error correction stream and obtain a 
des,red amount o, the error correction information needed to replace packets in an 
associated source data stream that were lost during ,he transmission even if some of 
the error correction stream packets themselves are lost in transmission. 

85. The system of Claim 84, wherein the program module for multicasting 
streams of packetized error correction information comprises a sub-module for 
incorporating an identifier into each packet in each of the streams of packetized error 
correction information wfrerein a dtfferen. identifier is employed for each of fhe error 
correction information streams, thereby altowing each receiver to select the number 
of error correction information streams to be received by specifying that only packets 
having particular identifiers be routed over the network to the receiver. 

86. The system of Claim 85. further comprising a program module for 
allowing a receiver to improve the qualNy of the received b ro adcas, by first selecting 
a desired number of source data streams while leaving enough bandwidth available 
to also select a number of error correction information streams for one or more of the 
source streams that will compensate, at teas, partially, for an inherent packet loss 
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rate associated with the receiver's connection to the network, and then selecting said 
number of error correction information streams. 

87. An error correction system for multicasting real-time media over a 
5 heterogeneous packet network to a plurality of receivers, comprising: 

a general purpose computing device; 

a computer program comprising program modules executable by the 
computing device, wherein the computing device is directed by the program modules 
of the computer program to, 

10 for ™ a series of source data streams from a media signal, said 

source data streams comprising a base layer representing the media signal at a 
least acceptable level of quality, and at least one enhancement layer which when 
combined with the base layer improves the level of quality of the media signal 
derivable therefrom, 

15 for each source data stream, create at least one error correction 

information stream from the source layer, wherein each error correction stream 
comprises a delayed version of the stream of source data associated therewith and 
each error correction stream associated with the same source data stream is 
progressively delayed from one another, 

20 - respectively multicast each source layer and error correction 

layer to a different network address for routing on to a receiver, and 

for each receiver, receive at least one source data stream and 
periodically receiving at least one error correction stream, and reconstructing a 
media signal from the at least one source data stream after having used the at least 

25 one error correction stream to recover source data packets lost in transmission of the 
source data stream. 

88. The system of Claim 87 wherein the receiving program module 
comprises sub-modules for: 

30 subscribing over the network to at least one source data stream; 

for each subscribed to source data stream, 

(i) monitoring the incoming source data stream for a missing 

data packet; 
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(ii) whenever a missing data packet is detected, subscribing 
to the least delayed version of the error correction streams associated with the 
source data stream, 

(iii) attempting to recover the missing source data stream 
5 packet from the subscribed to error correction stream, 

(iv) unsubscribing from the subscribed to error correction 

stream, 

(v) replacing the missing source data stream packet using 
the recovered packet whenever the missing packet is recovered from the subscribed 

10 to error correction stream, 

(vi) repeating steps (i) through (v) for the duration of the 

multicast 

89. The system of Claim 87, wherein said receiving program module 
15 comprises, sub-modules for: 

subscribing over the network to at least one source data stream 
for each subscribed to source data stream, 

(i) monitoring the incoming Source data stream for a missing 

data packet; 

20 (ii) whenever a missing data packet is detected, subscribing 

to the least delayed version of the error correction streams associated with the 

source data stream available on the network, 

(iii) attempting to recover the missing source data stream 

packet from the subscribed to error correction stream, 
25 (iv) unsubscribing from the currently subscribed to error 

correction stream, 

(v) whenever the missing packet cannot be recovered from 
the previously subscribed to error correction stream, determining whether there is at 
least one more error correction stream associated with the source data stream 

30 available on the network for subscription 

(vi) whenever the is another error correction stream available, 
subscribing to the least delayed version of the error correction streams associated 
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with the source data stream available on the network, which has not been 
subscribed to before in connection with said missing data packet, 

(vii) repeating steps (iii) through (vi) until the missing packet is 
recovered or it is determined there are no more previously unsubscribed to error 

5 correction streams available on the network, 

(viii) whenever the missing packet is recovered from a 
subscribed to error correction stream, replacing the missing source data stream 
packet using the recovered packet , 

(vi) repeating steps (i) through (viii) for the duration of the 

10 multicast. 

90. The system of Claim 88, wherein the sub-module for subscribing to the 
least delayed version of the error correction streams associated with the source data 
stream comprises sub-modules for: 

15 determining a currently available bandwidth associated with the 

receiver's connection to the network; 

unsubscribing from a subscribed to higher level source data stream to 
subscribe to said least delayed version of the error correction stream whenever there 
is not sufficient bandwidth available to subscribe to both. 

20 

91 . "The system of Claim 90, further comprising a sub-module for re- 
subscribing to said higher-level source data stream, whenever said least delayed 
version or the error correction stream is unsubscribed to thereby freeing up the 
bandwidth needed to re-subscribe to the higher-level source data stream. 

25 

92. An error correction system for multicasting real-time media over a 
heterogeneous packet network to a plurality of receivers, comprising: 

a general purpose computing device; 

a computer program comprising program modules executable by the 
30 computing device, wherein the computing device is directed by the program modules 
of the computer program to, 

form a series of source data streams from a media signal, said 
source data streams comprising a base layer representing the media signal at a 
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leas, .acceptable level of quality, and a, leas, one enhancement layer which when 
ompnses parity packets encoded from packets of the source data ,r«h 

layer to a d«» . reSPeC *' Vely mul «^«ng each source layer and error correction 
layer to a deferent network address for routing on to a receiver. 

for each receiver, receive at least one of said source data 
stream and receiving, a, leas, periodica,*, said a, leas, one error correction ,1, 
and reconstructing a media signal from the a, leas, one source data s^m X 
avtng used said a, teas, one error conection stream ,o recover source daTa plets 
lost ,n transmtssion of me source date stream. 

93. The system o, Cairn 92, wherein mere are multipte error correction 

r,:~: i,h each source data ^ - — - -~ 
com P nr:::i r correc " on in,omation — <™ - — >- 

(a) partitioning the source data stream into a plurality of data blocks 
each containing k data packets; a .iy or data blocks 

(b) selecting a previously unselected data block- 
to the r-hn ^ aPP ' yih9 " *"* (FEC > enc °<*ng technique 
to the chosen data block to produce n-k parity packets for that block where n > k 

stream; aSS '' 9nln9 "** **** t0 3 different error Action ' 

(e) repeating steps (b) through (d) for each data block and 

(f) ass.gning each error correction stream to one of W waves. 
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94. The system of Claim 93, wherein the multicasting program module 
comprises sub-modules for: 

multicasting all the error correction streams forming a first wave without 
delay in comparison to the source data stream; and 
5 multicasting the error correction streams in each subsequent wave with 

a delay in comparison to the immediately preceding wave. 

95. The system of Claim 94, wherein the receiving sub-module comprises, 
sub-modules for: 

10 subscribing over the network to at least one source data stream; 

subscribing to at least one error correction information stream 
associated with the first wave of error correction streams for each subscribed to 
source data stream; 

for each subscribed to source data stream, 
15 (0 receiving source data stream packets comprising a next 

incoming source data block of the subscribed to source data stream; 

(ii) receiving parity packets associated with the incoming 
source data block in the subscribed to error correction information stream or streams 
in the first wave, 

20 ('") determining whether the total number of source data 

stream packets and parity packets received for the incoming source data block is at 
least equal the number of source data stream packets multicast for the block; 

(iv) whenever the total number of source data stream packets 
and parity packets received for the incoming source data block does not equal or 

25 exceed the number of source data stream packets multicast for the block, 

determining whether there are any more waves available for subscription on the 
network, 

(v) if there are one or more waves available, subscribing to 
at least one error correction stream in a previously unsubscribed to wave having the 

30 least delay among the available waves, and attempting to capture the number of 
parity packets associated with the incoming data block which would cause the total 
number of source data stream packets and parity packets received for the incoming 
source data block to equal or exceed the number of source data stream packets 
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multicast for the block, and thereafter unsubscribing from said at least one error 
correction stream associated with the delayed wave, 

(vi) repeating step (iii) through (v) until the total number of 
source data stream packets and parity packets received for the incoming source 

5 data block equals or exceeds the number of source data stream packets multicast 
for the block, or it is determined there are no more waves available, 

(vii) whenever the total number of source data stream packets 
and parity packets received for the incoming source data block equals or exceeds 
the number of source data stream packets multicast for the block, recovering all the 

10 source data stream packets missing in the incoming source data block using the 
received parity packets, 

(viii) replacing said missing source data stream packets using 
the recovered packets, and 

(ix) repeating steps (i) through (viii) for each successive 
15 incoming source data block. 

96. The system of Claim 95, further comprising, prior to executing the sub- 
module for assigning each error correction stream to N one of W waves, a sub-module 
for collapsing at least some of the error correction streams such that all the parity 
20 packets associated with these error correction streams are sequentially included 
within the collapsed error correction stream, and wherein the sub-module for 
subscribing to any error correction stream includes a sub-module for subscribing to a 
collapsed error correction stream, as desired. 

25 97 • Tn e system of Claim 95, wherein the sub-module for subscribing over 

the network to at least one source data stream and subscribing to at least one error 
correction information stream associated with the first wave of error correction 
streams, comprises sub-modules for: 

determining a currently available bandwidth associated with the 

30 receiver's connection to the network; 

determining a current packet loss rate associated with receiving data 
over the network; 
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selecting the number of source data streams and the number of 
associated error correction streams which are to be received so as to allow the best 
possible level of quality in a media signal reconstructed from received source data 
streams, wherein said selecting is based on the currently available bandwidth and the 
current packet loss rate; and 

subscribing to each selected source data stream and each selected 
error correction information stream. 

98. The system of Claim 95, wherein the sub-module for subscribing to at 
least one error correction stream in a previously unsubscribed to wave, comprises 
the sub-module for: 

determining a currently available bandwidth associated with the 
receiver's connection to the network; 

discontinuing reception of a higher level source data stream to receive 
15 the at least one error correction stream whenever there is not sufficient bandwidth 
available to receive both. 

99. The system of Claim 94, wherein said receiving program module 
comprises, sub-modules for; 

subscribing over the network to at least one source data stream; 
for each subscribed to source data stream, 

(i) receiving source data stream packets comprising a next 
incoming source data block of the subscribed to source data stream; 

(ii) . determining whether any of the source data stream 
packets in the incoming source data block were lost during transmission; 

(iii) whenever it is determined that any source data stream 
packets were lost during transmission of the incoming source data block , 
determining whether there are error correction stream available for subscription on 
the network, 

(iv) if there are one or more waves available, subscribing to 
at least one error correction stream in a previously unsubscribed to wave having the 
least delay among the available waves, and attempting to capture the number of 
parity packets associated with the incoming data block which would cause the total 
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number of source data stream packets and parity packets received for the incoming 
source data block to equal or exceed the number of source data stream packets 
multicast for the block- and thereafter unsubscribing from said at least one error 
correction stream associated with the delayed wave, 
5 (v) repeating step (iv) until the total number of source data 

stream packets and parity packets received for the incoming source data block 
equals or exceeds the number of source data stream packets multicast for the block 
or there are no more waves available for subscription, 

(vi) whenever the total number of source data stream packets 
10 and parity packets received for the incoming source data block equals or exceeds 

the numbar of source data stream packets multicast for the block, recovering all the 
source data stream packets missing in the incoming source data block using the 
received parity packets, 

(vii) replacing said missing source data stream packets using 
15 the recovered packets, and 

(ix) repeating steps (i) through (viii) for each successive 
incoming source data block. 

100. The system of Claim 99, further comprising executing, prior to the sub- 
20 module for assigning each error correction stream to one of W waves, a sub-module 

for collapsing at least some of the error correction streams such that all the parity 
packets associated with these error correction streams are sequentially included 
within the collapsed error correction stream, and wherein the sub-modules for 
subscribing to any errpr correction stream include subscribing to a collapsed error 
25 correction stream, as desired. 

101 . The system of Claim 99, wherein the sub-module for subscribing to at 
least one error correction stream in a previously unsubscribed to wave, comprises 
sub-modules for: 

30 determining a currently available bandwidth associated with the 

receiver's connection to the network; 
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discontinuing reception of a higher level source data stream to receive 
the at least one error correction stream whenever there is not sufficient bandwidth 
available to receive both. 

5 1 02. A computer-readable memory for use in a receiver-driven layered 

multicast of real-time media over a heterogeneous packet network to a plurality of 
receivers, wherein the real-time media is transmitted over the network in multiple 
streams of packetized source data forming hierarchical layers of information, 
comprising: 

10 a computer-readable storage medium; and 

a computer program comprising program modules stored in the storage 
medium, wherein the storage medium is so configured by the computer program that 
it causes the computer to, 

produce at least one stream of packetized error correction 
15 information for at least one of the streams of packetized source data, wherein each . 
error correction stream comprises a delayed version of the stream of packetized 
source data associated therewith, 

multicast the streams of packetized error correction information 
to said receivers to allow each receiver to replace packets of source data lost during 
20 transmission. 

1 03. The computer-readable memory of Claim 1 02 wherein the program 
module for multicasting streams of packetized error correction information comprises 
sub-modules for: 

25 associating each stream of packetized error correction information with 

a one of the multiple streams of packetized source data; and 

incorporating in each error correction stream information necessary to 
assist a receiver in recovering said packets of lost source data belonging to the 
particular stream of packetized source data associated with the error correction 

30 stream. 

104. The computer-readable memory of Claim 103, wherein there are 
multiple streams of error correction information associated with each stream of 

77 



3CID: <WO 00761 13A1J_> 



WO 00/761 13 PCT/US00/14060 

source data, and wherein the incorporating sub-module comprises a sub-module for 
incorporating error correction information in each error correction stream associated 
with the same source data stream which makes the streams redundant, said 
redundant streams making it possible for a receiver to employ more than one error 
correction stream and obtain a desired amount of the error correction information 
needed to replace packets in an associated source data stream that were lost during 
the transmission even if some of the error correction stream packets themselves are 
lost in transmission. 

105. The computer-readable memory of Claim 104, wherein the program 
module for multicasting streams of packetized error correction information comprises 
a sub-module for incorporating an identifier into each packet in each of the streams 
of packetized error correction information wherein a different identifier is employed 
for each of the error correction information streams, thereby allowing each receiver 
to select the number of error correction information streams to be received by 
specifying that only packets having particular identifiers be routed over the network 
to the receiver. 

j 

\ 

1 06. The computer-readable memory of Claim 1 05, further comprising a 
program module for allowing a receiver to improve the quality of the received 
broadcast by first selecting a desired number of source data streams while leaving 
enough bandwidth available to also select a number of error correction information 
streams for one or more of the source streams that will compensate, at least 
partially, for an inherent packet loss rate associated with the receiver's connection to 
the network, and then selecting said number of error correction information streams. 

107. A computer-readable memory for multicasting real-time media over a 
heterogeneous packet network to a plurality of receivers, comprising: 

a computer-readable storage medium; and 

a computer program comprising program modules stored in the storage 
medium, wherein the storage medium is so configured by the computer program that 
it causes the computer to, 
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form a series of source data streams from a media signal, said 
source data streams comprising a base layer representing the media signal at a 
least acceptable level of quality, and at least one enhancement layer which when 
combined with the base layer improves the level of quality of the media signal 
5 derivable therefrom, 

for each source data stream, create at least one error correction 
information stream from the source layer, wherein each error correction stream 
comprises a delayed version of the stream of source data associated therewith and 
each error correction stream associated with the same source data stream is 
10 progressively delayed from one another, 

respectively multicast each source layer and error correction 
layer to a different network address for routing on to a receiver, and 

for each receiver, receive at least one source data stream and 
periodically receiving at least one error correction stream, and reconstructing a 
15 media signal from the at least one source data stream after having used the at least 
one error correction stream to recover source data packets lost in transmission of the 
source data stream. 

108. The computer-readable memory of Claim 107 wherein the receiving 
20 program module comprises sub-modules for: 

subscribing over the network to at least one source data stream; 
for each subscribed to source data stream, 

(i) monitoring the incoming source data stream for a missing 

data packet; 

25 (») whenever a missing data packet is detected, subscribing 

to the least delayed version of the error correction streams associated with the 
source data stream, 

(iii) attempting to recover the missing source data stream 
packet from the subscribed to error correction stream, 

30 (iv) unsubscribing from the subscribed to error correction 

stream, 
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(v) replacing the missing source data stream * • 



5 multicast. 



(v.) repeating steps (i) through (v) for the duration of the 
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repeating steps (i) through (viii) for the duration of the 



110. The computer-readable memory of Claim 108, wherein the sub-module 
5 for subscribing to the least delayed version of the error correction streams 

associated with the source data stream comprises sub-modules for: 

determining a currently available bandwidth associated with the 
receiver's connection to the network; 

unsubscribing from a subscribed to higher level source data stream to 
10 subscribe to said least delayed version of the error correction stream whenever there 
is not sufficient bandwidth available to subscribe to both. 

111. The computer-readable memory of Claim 1 1 0, further comprising a 
sub-module for re-subscribing to said higher-level source data stream, whenever 

15 said least delayed version of the error correction stream is unsubscribed to thereby 
freeing up the bandwidth needed to re-subscribe to the higher-level source data 
stream. 

112. A computer-readable memory for multicasting real-time media over a 
20 heterogeneous packet network to a plurality of receivers, comprising: 

a computer-readable storage medium; and 

a computer program comprising program modules stored in the storage 
medium, wherein the storage medium is so configured by the computer program that 
it causes the computer to, 

25 form a series of source data streams from a media signal, said 

spurce data streams comprising a base layer representing the media signal at a 
least acceptable level of quality, and at least one enhancement layer which when 
combined with the base layer improves the level of quality of the media signal 
derivable therefrom, 

30 for each source data stream, create at least one error correction 

information stream from the source layer, wherein each error correction stream 
comprises parity packets encoded from packets of the source data stream which 
have been subjected to a linear transform, said linear transform producing a series of 
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parity packets which can be employed to assist in the recovery of lost source data 
packets using a reverse transform, and wherein at least one of said at least one error 
correction information streams is delayed in comparison to the source data stream, 

respectively multicasting each source layer and error correction 
layer to a different network address for routing on to a receiver, 

for each receiver, receive at least one of said source data 
stream and receiving, at least periodically, said at least one error correction layer, 
and reconstructing a media signal from the at least one source data stream after 
having used said at least one error correction stream to recover source data packets 
lost in transmission of the source data stream. 

113. The computer-readable memory of Claim 112, wherein there are 
multiple error correction streams associated with each source data stream, and 
wherein the program module for creating at least one error correction information 
stream from the source layer, comprises sub-modules for: 

(a) partitioning the source data stream into a plurality of data blocks 
each containing k data packets; 

(b) selecting a previously unselected data block; 

(c) applying a Forward Error Correction (FEC) encoding technique 
to the chosen data block to produce n-k parity packets for that block where n > k; 

(d) assigning each parity packet to a different error correction 

stream; 

(e) repeating steps (b) through (d) for each data block; and 

(f) assigning each error correction stream to one of W waves. 

114. The computer-readable memory of Claim 1 1 3, wherein the multicasting 
program module comprises sub-modules for: 

multicasting all the error correction streams forming a first wave without 
delay in comparison to the source data stream; and 

multicasting the error correction streams in each subsequent wave with 
a delay in comparison to the immediately preceding wave. 
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1 1 5. The computer-readable memory of Claim 1 14, wherein the receiving 
sub-module comprises, sub-modules for: 

subscribing over the network to at least one source data stream; 
subscribing to at least one error correction information stream 
5 associated with the first wave of error correction streams for each subscribed to 
source data stream; 

for each subscrbed to source data stream, 

(i) receiving source data stream packets comprising a next 
incoming source data block of the subscribed to source data stream; 
10 (") receiving parity packets associated with the incoming 

source data block in the subscribed to error correction information stream or streams 
in the first wave; 

(iii) determining whether the total number of source data 
stream packets and parity packets received for the incoming source data block is at 

15 least equal the number of source data stream packets multicast for the block; 

(iv) whenever the total number of source data stream packets 
and parity packets received for the incoming source data block does not equal or 
exceed the number of source data stream packets multicast for the block, 
determining whether there are any more waves available for subscription on the 

20 network, 

(v) if there are one or more waves available, subscribing to 
at least one error correction stream in a previously unsubscribed to wave having the 
least delay among the available waves, and attempting to capture the number of 
parity packets associated with the incoming data block which would cause the total 

25 number of source data stream packets and parity packets received for the incoming 
source data block to equal or exceed the number of source data stream packets 
multicast for the block, and thereafter unsubscribing from said at least one error 
correction stream associated with the delayed wave, 

(vi) repeating step (iii) through (v) until the total number of 
30 source data stream packets and parity packets received for the incoming source 

data block equals or exceeds the number of source data stream packets multicast 
for the block, or it is determined there are no more waves available, 
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(vii) whenever the total number of source data stream packets 
and parity packets received for the incoming source data block equals or exceeds 
the number of source data stream packets multicast for the block, recovering all the 
source data stream packets missing in the incoming source data block using the 
received parity packets, 

(viii) replacing said missing source data stream packets using 
the recovered packets, and 

(ix) repeating steps (i) through (viii) for each successive 
incoming source data block. 

1 1 6. The computer-readable memory of Claim 1 1 5, further comprising, prior 
to executing the sub-module for assigning each error correction stream to one of W 
waves, a sub-module for collapsing at least some of the error correction streams 
such that all the parity packets associated with these error correction streams are 
sequentially included within the collapsed error correction stream, and wherein the 
sub-module for subscribing to any error correction stream includes a sub-module for 
subscribing to a collapsed error correction stream, as desired. 

1 1 7. The computer-readable memory of Claim 1 1 5, wherein the sub-module 
for subscribing over the network to at least one source data stream and subscribing 
to at least one error correction information stream associated with the first wave of 
error correction streams, comprises sub-modules for: 

determining a currently available bandwidth associated with the 
receiver's connection to the network; 

determining a current packet loss rate associated with receiving data 
over the network; 

selecting the number of source data streams and the number of 
associated error correction streams which are to be received so as to allow the best 
possible level of quality in a media signal reconstructed from received source data 
streams, wherein said selecting is based on the currently available bandwidth and the 
current packet loss rate; and 

subscribing to each selected source data stream and each selected 
error correction information stream. 



84 



WO 00/76113 PCT/USO0/14060 

1 1 8. The computer-readable memory of Claim 1 1 5, wherein the sub-module 
for subscribing to at least one error correction stream in a previously unsubscribed to 
wave, comprises the sub-module for: 

determining a currently available bandwidth associated with the 
receivers connection to the network; 

discontinuing reception of a higher level source data stream to receive 
the at least one error correction stream whenever there is not sufficient bandwidth 
available to receive both. 

1 1 3. The computer-readable memory of Claim 1 1 4, wherein said receiving 
program module comprises, sub-modules for: 

subscribing over the network to at least one source data stream; 
for each subscribed to source data stream, 

(i) receiving source data stream packets comprising a next 
incoming source data block of the subscribed to source data stream; 

(ii) determining whether any of the source.data stream 
packets in the incoming source data block were lost during transmission; 

(iii) whenever it is determined that any source data stream 
packets were lost during transmission of the incoming source data block , 
determining whether there are error correction stream available for subscription on 
the network, 

(iv) if there are one or more waves available, subscribing to 
at least one error correction stream in a previously unsubscribed to wave having the 
least delay among the-available waves, and attempting to capture the number of 
parity packets associated with the incoming data block which would cause the total 
number of source data stream packets and parity packets received for the incoming 
source data block to equal or exceed the number of source data stream packets 
multicast for the block, and thereafter unsubscribing from said at least one error 
correction stream associated with the delayed wave, 

(v) repeating step (iv) until the total number of source data 
stream packets and parity packets received for the incoming source data block 
equals or exceeds the number of source data stream packets multicast for the block 
or there are no more waves available for subscription, 



85 



WO 00/76113 

PCT/US00/14060 
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